[WIP] 5 - Transmutes by jchlanda · Pull Request #159082 · rust-lang/rust

jchlanda · 2026-07-10T13:23:48Z

Implement pointer authentication domain handling for function pointer transmutes. When function pointer type discrimination is enabled, transmuting between function pointer types with different authentication domains now re-signs the pointer using the appropriate discriminator.

This is part 5 of a sequence of 8 PRs, that together aim to bring function pointer type discrimination support:

Useful links:

Previous PAC work:
- pauthtest introduction: Introduce aarch64-unknown-linux-pauthtest target #155722
- Library support follow up: Library support for aarch64-unknown-linux-pauthtest target #156548
- Config follow up: Pointer authentication config and user facing options #156712
Project goal and tracking issue
Clang's implementation of ASTContext::encodeTypeForFunctionPointerAuth
LLVM's SpiHash

This patch implements Rust's equivalent of Clang's function pointer type discriminator computation used in pointer authentication. Compatibility with Clang is a primary goal. The discriminator produced for a given external "C" function type must match the value computed by Clang so that function pointers can be exchanged safely between Rust and C code while preserving pointer authentication semantics. The implementation mirrors Clang's behavior in `ASTContext::encodeTypeForFunctionPointerAuth`, ensuring that identical C-compatible function types produce identical discriminators. See: <https://clang.llvm.org/doxygen/ASTContext_8cpp.html#abb1375e068e807917527842d05cadea3>.

kovdan01 · 2026-07-20T08:46:22Z

+        bx: &mut Bx,
+        val: Bx::Value,
+        info: TransmuteInfo<'tcx>,
+    ) -> Bx::Value {


Do we only expect this function being called if function pointer type discrimination is enabled? If so, would it be beneficial to add an assertion ensuring this and making this pre-condition explicit?

View changes since the review

Sure, assert added.

kovdan01 · 2026-07-20T08:52:57Z

+                    // This is the address of a compiler-generated TLS shim function. It is not an
+                    // externally visible function pointer and does not require function pointer
+                    // authentication signing.
+                    let fn_ptr = bx.get_fn_addr(instance, None);


Could you please refer me to a test which covers this? And also, do we end up with indirect call to this function somewhere, or is it only called directly? If this is called indirectly, the function pointer must be signed and the call must be authenticated anyway. The fact that the function is compiler-generated does not mean that the attacker cannot substitute its address - they can.

I'm just not sure if I'm interpreting your comments here correctly, so I'd like to take a look at the test :)

View changes since the review

The function pointer here is not a user-provided function pointer. The callee is constructed explicitly as:

let instance = ty::Instance { def: ty::InstanceKind::Shim(ty::ShimKind::ThreadLocal(def_id)), args: ty::GenericArgs::empty(), };

My reading is that it asks the compiler to create a TLS accessor shim associated with the thread local static identified by def_id. The user only provides the thread local static; they do not provide the function that is called.

The "compiler-generated" part of the comment was meant as a shorthand for the fact that this is a rustc-internal TLS shim, not an arbitrary extern "C" function pointer exposed by the program.

Regarding the test, I initially tried to cover this with a cross-crate TLS static, something along the lines of:

aux crate:

#![feature(thread_local)] #[thread_local] pub static TLS: u32 = 42;

main crate:

extern crate tls; #[no_mangle] pub unsafe fn read_tls() -> u32 { tls::TLS }

However, this only reaches the normal TLS lowering path for our target. The shim Instance creation path is guarded by:

!def_id.is_local() && tcx.needs_thread_local_shim(def_id)

which basically boils down to only Windows/MSVC, making this kinda irrelevant (?).

The function pointer here is not a user-provided function pointer.

Yeah, but is it still called indirectly? Pointer authentication is intended to protect us from memory-corruption-induced vulnerabilities, and while in "correct" flow we are expecting this non-user-provided pointer be intact, with some memory corruption exploited by attacker, this pointer might be substituted even if the pointer by itself is "normally" non-user-provided (we are already in abnormal situation).

Please let me know if I've misunderstanding smth

which basically boils down to only Windows/MSVC, making this kinda irrelevant (?).

Hmm, if that's correct, maybe we can add an assertion ensuring that ptrauth-calls is not expected to be requested here with a comment that only Windows/MSVC path is expected to reach this place, while these are deliberately out of scope now?

Yeah, but is it still called indirectly? Pointer authentication is intended to protect us from memory-corruption-induced vulnerabilities, and while in "correct" flow we are expecting this non-user-provided pointer be intact, with some memory corruption exploited by attacker, this pointer might be substituted even if the pointer by itself is "normally" non-user-provided (we are already in abnormal situation).

I get that, all I'm saying is that compiler will not generate an extern "C" for the shim, so we don't have to worry about it (or at least not in the current scope). But to answer your question, looking at get_fn implementation I think it will be a direct call that would be issued.

Hmm, if that's correct, maybe we can add an assertion ensuring that ptrauth-calls is not expected to be requested here with a comment that only Windows/MSVC path is expected to reach this place, while these are deliberately out of scope now?

OK, assert/comment added.

that compiler will not generate an extern "C" for the shim

Got it now, thanks for explanation!

kovdan01 · 2026-07-22T09:55:42Z

+    ///
+    /// Only immediate values are subject to ptrauth adjustment; other
+    /// representations are passed through unchanged.
+    fn codegen_semantic_transmute_place(


Is my understanding correct that this method is ptrauth-specific? If so, would it be worth naming it accordingly so it does not look like a generic method?

Also, self.codegen_semantic_transmute_place(bx, src, dst); looks as its only use and it's an "alternative" to src.store_with_annotation(bx, dst.val.with_type(src.layout)); when type discrimination is available. Two questions arise:

Could this be made more "symmetric" to original store_with_annotation? If not, could you please explain why? It just might look slightly counter-intuitive to call for routines doing the same thing (in two different cases) but calling them differently.

If answer to p.1 is "yes" (we can make it symmetric) - maybe we can even hoist all this logic to store_with_annotation (probably under some checks against type discrimination being enabled)?

I suppose that the current version tries to be "symmetric" with codegen_semantic_transmute_operand and I'm not sure which approach is better - to keep things as implemented now (and asymmetry with store_with_annotation) or vice versa

View changes since the review

Is my understanding correct that this method is ptrauth-specific? If so, would it be worth naming it accordingly so it does not look like a generic method?

Yes, this is ptrauth-specific function. The "semantic" naming was intended to distinguish it from the existing transmute path (bitwise path), but I agree that it is not a general semantic transmute mechanism and the name is misleading. I have renamed it to be more explicit: ptrauth_codegen_transmute_place (and similarly ptrauth_codegen_transmute_operand).

Could this be made more "symmetric" to original store_with_annotation? If not, could you explain why?

I think keeping this separate from store_with_annotation is preferable.

To me, store_with_annotation represents a normal MIR store operation: give me a value and I'll store it in the destination location. The ptrauth "adjustment" here is not a property of storing the value; it is a property of changing the type of a function pointer during a transmute - and so requiring a resign.

Also, from a quick glance, store_with_annotation is used in many places, moving this into that function would make a lot of stores perform redundant ptrauth checks (that assumes that the information is available - best case scenario, I've not checked it, but we might need to adjust it to provide the types - worst case scenario).

The current split mirrors the two existing transmute lowering paths:

SSA values

destination places
so I think keeping the ptrauth logic at that (higher) level serves us better.

kovdan01 · 2026-07-22T10:01:25Z

+        let OperandValue::Immediate(ptr) = val else {
+            return val;
+        };
+
+        let info = TransmuteInfo { src_ty: operand.layout.ty, dst_ty: cast.ty };
+
+        OperandValue::Immediate(self.resign_transmuted_fn_ptr(bx, ptr, info))


Is it better to use this separate codegen_semantic_transmute_operand method for this logic, or can we put it into codegen_transmute_operand under some condition against type discrimination being enabled?

I do not have a strong preference and I guess there might be reasons to have a separate method - but I'm just wondering if we can simplify call sites and always use the same "default" method.

View changes since the review

See above, I like the explicit split that mirrors how transmutes are handled. My preference would be to keep it the way it is now (after function name changed): ptrauth_codegen_transmute_operand and ptrauth_codegen_transmute_place.

kovdan01 · 2026-07-22T10:04:49Z

+    /// function pointer or function definition.
+    ///
+    /// Returns the corresponding layout if one is found, otherwise `None`.
+    fn transparent_fn_ptr_layout(


BTW, does rust have smth like C++'s private:? This only looks used in codegen_semantic_transmute_place and maybe we can avoid exposing this to external users

View changes since the review

transparent_fn_ptr_layout has no visibility modifier, so it is private (pub is required for public visibility).

kovdan01 · 2026-07-22T10:05:38Z


+/// Type metadata used when applying pointer authentication semantics during
+/// transmute lowering.
+struct TransmuteInfo<'tcx> {


Nit: Is this struct intended to be used in this file only? If yes, does rust have some way to indicate that (e.g. like anonymous namespaces in C++)?

View changes since the review

At module scope (.rs file) a struct definition, like above, is private to the containing module by default. And because it is not defined as pub other modules can not import it.

kovdan01 · 2026-07-22T10:06:44Z

+    ///
+    /// A discriminator value of `0` is used to represent non-function-pointer
+    /// or "raw pointer domain" values:
+    /// ```text


Nit: do we really want text here? Maybe rust or just no language annotation? Or is text the intentional choice?

View changes since the review

It is, if it was rust tidy would try to lift that snippet and try to compile it - and fail :) .

kovdan01 · 2026-07-22T10:09:04Z

+        OperandValue::Immediate(self.resign_transmuted_fn_ptr(bx, ptr, info))
+    }
+
+    /// Applies pointer-authentication domain correction for a function pointer


Do we need to introduce the term "domain" here (or was it already introduced previously)? I'm just concerned that more terms -> more potential inconsistencies, misunderstanding, etc. Especially given that in this particular context, it seems that terms "domain" and "function pointer type discriminator" are used as synonyms (please let me know if I'm missing smth)

View changes since the review

Fair point, we don't. Reworded.

kovdan01 · 2026-07-22T10:11:42Z

+    /// A discriminator value of `0` is used to represent non-function-pointer
+    /// or "raw pointer domain" values:


I do not honestly remember, so could you please remind me: in Clang (with function pointer type discrimination enabled), if we cast signed type-discriminated C function pointer (IA key, non-zero discriminator) to a non-function pointer (e.g. void*), what do we end up with? Would it be a pointer signed with IA key and zero discriminator?

View changes since the review

You mean something like this, right?

typedef void (*fptr_t)(void); fptr_t fptr; void* test1() { return (void*)fptr; }

This results in a resign to 0-discrimination:

%5 = call i64 @llvm.ptrauth.resign(i64 %4, i32 0, i64 18983, i32 0, i64 0)

kovdan01 · 2026-07-22T10:17:44Z

-                    let fn_ptr =
-                        bx.get_fn_addr(instance, bx.sess().pointer_authentication_functions());
+                    // This is the address of a compiler-generated TLS shim function. It is not an
+                    // externally visible function pointer and does not require function pointer


Would it be worth to somehow slightly re-phrase "externally visible" here? As we've discussed, what is really meant is that it's not ExternAbi::{C|System} which are the only ones supported by pointer authentication.

My concern is that now the phrase might be understood as "it's compiler-internal pointer not exposed to users" (and it's true), but it's not sufficient to decide that pointer authentication is not needed. What is sufficient - is that only C/System extern ABI imply use of pointer authentication.

Or, probably even better. We already have an assertion against pointer authentication being disabled a couple of lines above. It is a sufficient reason to have None here w/o any further explanation. Maybe worth just move the assertion to this line so the narrative is clear "why None? because of ^^^"

View changes since the review

kovdan01 · 2026-07-22T10:19:53Z

An overall comment regarding transmute - do we have tests only for std::mem::transmute or do we ensure that the logic from this PR would work properly even for "custom" user-provided transmute alternatives (which are "semantic" transmutes)?

This patch introduces the following: * Extends `FnAbi` (`callconv`) with a `ptrauth_type_discriminator` field. This field is only used when emitting pointer authentication call bundles. It is stored in `FnAbi` because the call site is not guaranteed to have access to an `Instance`, so the discriminator cannot always be computed on demand. * Adds support for `llvm.ptrauth.resign`. This intrinsic will be used when support for semantic transmute is added. * Performs a minor API redesign as groundwork for allowing call sites to modify schemas in place.

Also remove error messages/tests that used to guarded it.

The codegen now walks the layout of static initializer types to find extern "C" function pointer fields, computes their type discriminators, and applies those discriminators when emitting authenticated function pointer relocations. Also make sure that type discrimination is never applied to init/fini entries.

Implement pointer authentication domain handling for function pointer transmutes. When function pointer type discrimination is enabled, transmuting between function pointer types with different authentication domains now re-signs the pointer using the appropriate discriminator.

jchlanda · 2026-07-22T15:28:26Z

An overall comment regarding transmute - do we have tests only for std::mem::transmute or do we ensure that the logic from this PR would work properly even for "custom" user-provided transmute alternatives (which are "semantic" transmutes)?

The implementation doesn't recognize std::mem::transmute specifically. It runs during codegen for MIR mir::CastKind::Transmute | mir::CastKind::Subtype, so it's driven by the MIR representation rather than by the source-level implementation.

This comment has been minimized.

Sign in to view

jchlanda force-pushed the jakub/pac_ty_disc_PR_5 branch 2 times, most recently from dc131ff to 589c793 Compare July 14, 2026 07:28

This comment has been minimized.

Sign in to view

jchlanda force-pushed the jakub/pac_ty_disc_PR_5 branch from 589c793 to 49f9b55 Compare July 14, 2026 08:54

This comment has been minimized.

Sign in to view

jchlanda force-pushed the jakub/pac_ty_disc_PR_5 branch from 49f9b55 to ec37937 Compare July 17, 2026 07:05

This comment has been minimized.

Sign in to view

jchlanda force-pushed the jakub/pac_ty_disc_PR_5 branch from ec37937 to 0624ea5 Compare July 17, 2026 08:22

jchlanda force-pushed the jakub/pac_ty_disc_PR_5 branch from 0624ea5 to de79b6a Compare July 17, 2026 09:48

kovdan01 reviewed Jul 20, 2026

View reviewed changes

jchlanda force-pushed the jakub/pac_ty_disc_PR_5 branch 4 times, most recently from f19ee3f to 03410f3 Compare July 21, 2026 13:19

kovdan01 reviewed Jul 22, 2026

View reviewed changes

jchlanda added 5 commits July 22, 2026 12:20

[PAC] Add unit tests for sip hash

03337eb

[PAC] Enable support for FPTR_TYPE_DISCR in ABI Version

af72799

Also remove error messages/tests that used to guarded it.

jchlanda force-pushed the jakub/pac_ty_disc_PR_5 branch from 03410f3 to 8126570 Compare July 22, 2026 14:57

		/// A discriminator value of `0` is used to represent non-function-pointer
		/// or "raw pointer domain" values:

Uh oh!

Uh oh!

Conversation

jchlanda commented Jul 10, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

kovdan01 Jul 20, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 20, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 22, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 22, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 22, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 22, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 22, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 22, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 22, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 Jul 22, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kovdan01 commented Jul 22, 2026

Uh oh!

jchlanda commented Jul 10, 2026 •

edited by rustbot

Loading

kovdan01 Jul 20, 2026 •

edited by rustbot

Loading

kovdan01 Jul 20, 2026 •

edited by rustbot

Loading

kovdan01 Jul 22, 2026 •

edited by rustbot

Loading

kovdan01 Jul 22, 2026 •

edited by rustbot

Loading

kovdan01 Jul 22, 2026 •

edited by rustbot

Loading

kovdan01 Jul 22, 2026 •

edited by rustbot

Loading

kovdan01 Jul 22, 2026 •

edited by rustbot

Loading

kovdan01 Jul 22, 2026 •

edited by rustbot

Loading

kovdan01 Jul 22, 2026 •

edited by rustbot

Loading

kovdan01 Jul 22, 2026 •

edited by rustbot

Loading