RFC: No (opsem) Magic Boxes

[chorman0773]

Summary

Currently, the operational semantics of the type alloc::boxed::Box<T> is in dispute, but the compiler adds llvm noalias to it. To support it, the current operational semantics models have the type use a special form of the Unique (Stacked Borrows) or Active (Tree Borrows) tag, which has aliasing implications, validity implications, and also presents some unique complications in the model and in improvements to the type (e.g. Custom Allocators). We propose that, for the purposes of the runtime semantics of Rust, Box is treated as no more special than a user-defined smart pointer you can write today¹. In particular, it is given similar behaviour on a typed copy to a raw pointer.

Rendered

Footnotes

1. We maintain some trivial validity invariants (such as alignment and address space limits) that a user cannot define, but these invariants only depend upon the value of the Box itself, rather than on memory. ↩

[clarfonthey]

It feels odd that one of the clear options is left out: why not expose a Unique<T> that has the same semantics as Box<T>, so any smart pointer type can use it?

Like, I definitely agree that Box shouldn't have special semantics that you can't reproduce elsewhere. But among the options, it feels pretty limiting to come to the conclusion that we should eliminate those semantics, rather than just making them reproducible elsewhere.

I agree that whatever happens shouldn't be specific to the Global allocator, though.

[scottmcm]

We maintain some trivial validity invariants (such as alignment and address space limits) that a user cannot define

I, at least, fully expect us to eventually have some way of writing alignment-obeying raw pointers in Rust in some way. If nothing else, slice::Iter is less good than it could be because of the lack of them, and optimizing that type is super-important.

(Transmuting between NonNull<T> and unsafe<'a> &'a T is one thing that might work, for example, though it's also possible that the opsem for that will end up saying that it doesn't and that something else is required instead.)

EDIT: added a word to try to communicate that I wasn't expecting this RFC to include such a type.

[chorman0773]

I, at least, fully expect us to have some way of writing alignment-obeying raw pointers in Rust in some way. If nothing else, slice::Iter is less good than it could be because of the lack of them, and optimizing that type is super-important.

That's my hope for the future as well, but to avoid the RFC becoming too cluttered, I am refraining from defining such a type in this RFC.

[juntyr]

Is there a list of optimisations that depend on noalias being emitted for Box’es?

[clarfonthey]

The RFC seems pretty clear that noalias hasn't really provided many benefits compared to being an extra burden to uphold for implementers, but maybe it is worth seeing if there are any sources that can provide a bit more detail on that.

[kennytm]

sorry but is the term "typed copy" explained somewhere?

(the explanations I could find are from pretty unofficial places like a reddit¹ and urlo² post)

Footnotes

1. "they are like memcpy, but the copy occurs with a type, giving the compiler some extra power." ↩

2. "a typed copy consists of essentially decoding the AM-bytes into the abstract value then encoding that abstract value back to AM-bytes at the new location." ↩

[chorman0773]

The urlo definition is probably good.
It's defined in the opsem, but I don't know if we have a very good written record of that other than spread arround zulip threads and github issues.

[scottmcm]

I'm a fan of this. I think that people moving from Vec<T> to Box<[T]> having to deal with drastically-different soundness rules is a giant footgun, and getting rid of the special [ST]B behaviour here sounds good to me.

[nikomatsakis]

My general take:

The two "endpoints" here are

• Efficient end-user abstractions: this allows most safe code to run faster. This would have strong alias requirements and would not expose raw/unsafe details. This permits non-obvious optimizations (e.g., small string optimization or 0-length capacity).
• Building blocks for unsafe code: this exposes raw/unsafe details.

From what I can tell, we current orient Box as the former but Vec and String as the latter. That seems backwards, since if anything those are far more useful as abstractions than Box is.

If I could go back in time, I think I would favor end-user abstractions and offer different types (e.g., RawVec or RawBuffer or something) that exposed their innards, but I think that ship has sailed, and we might as well embrace the current situation (which is nice in some ways too).

[nikomatsakis]

The RFC would benefit from some attempt to quantify the impact on performance, though our lack of standardized runtime benchmarks makes that hard.

[traviscross]

@rust-lang/opsem: We were curious in our discussions, does this RFC represent an existing T-opsem consensus?

[chorman0773]

We were curious in our discussions, does this RFC represent an existing T-opsem consensus?

It does not represent any FCP done by T-opsem, which is why I've included them here. The claims I make, including those about the impact on the operation semantics, are included in the request for comment and consensus.

The RFC would benefit from some attempt to quantify the impact on performance, though our lack of standardized runtime benchmarks makes that hard.

I recall some perf PR's (using the default rustc-perf suite) being done to determine the impact, which showed negligible impact. I can probably pull them up at some point in the RFC's lifecycle.

• [perf experiment] Don't emit noalias for box when compiling rustc itself rust#99527 is such PR, and showed only a secondary regression of 1.9% in Cycle Count, but an improvement in Max RSS of 4.0% on average.

[RalfJung]

The Reference has been adjusted a while ago to state validity invariants positively, i..e by listing what must be true, instead of listing what must not be false. IMO that's more understandable, and the RFC should be updated to also do that.

[RalfJung]

I agree that whatever happens shouldn't be specific to the Global allocator, though.

There are patterns of using a custom per-Box allocator that are incompatible with the aliasing requirements, at least under our current aliasing models. See rust-lang/miri#3341 for an example. So if we always make Box be unique, we have to declare those allocators to be UB.

Is there a list of optimisations that depend on noalias being emitted for Box’es?

It's "every LLVM optimization that looks at alias information". The question is how much that matters in practice, which is hard to evaluate.

We were curious in our discussions, does this RFC represent an existing T-opsem consensus?

As Connor said, not in any formal sense. Several opsem members have expressed repeatedly that they want to see noalias on Box go, but I don't know whether we have team consensus on this.

My own position is that I love how this simplifies the model and Miri, I am just slightly concerned about this being an irreversible loss of optimization potential that we might regret later. Absence of evidence of optimization benefit is not evidence of absence. Our benchmarks likely just don't have a lot of functions that take Box<T> by value. However, that in itself is an indication that the optimization benefit is likely limited.

Is there a way we can query the ecosystem for functions taking Box<T> by value?

[RalfJung]

This is actually the status quo, since rust-lang/rust#122018

[clarfonthey]

Just to follow up on some of the discussion, it wasn't immediately clear to me that types similar to Box, like Vec and Arc, genuinely don't have these semantics, even though it is implied by the fact that Box is unique in this regard. I think this is worth emphasising more in the text of the RFC, since in a very real sense, we've been going without these semantics for most^{[citation needed]} parts of Rust totally fine, which further emphasises less will be lost by removing it for Box.

I would still love if there's more data showing the lack of returns on noalias optimisations, since it feels wrong that something with a lot of history and usage isn't helping that much, but the fact that Vec and other types don't have this optimisation at least helps us understand that it's not going to cause any performance issues if removed.

[chorman0773]

I am just slightly concerned about this being an irreversible loss of optimization potential that we might regret later

FTR, speaking right now as one of the main developers of lccc, my opinion is that the best way to mitigate any loss of future optimization potential is to just be far more granular with &mut. I don't think spamming extra noalias on parameters is necessary if we just emit more metadata on derefs (llvm's scoped noalias and dereferenceable, and lccc's unique and dereferenceable type attributes). If there are problems with doing that, I don't think this is a place where we should necessarily bend the whole language to this one attribute on one backend, especially given the complexity of maintaining the feature as-is, where we keep running into fundamental issues with the special treatment of Box. If there are legitimate optimizations lost as a result of the function level noalias, that can't be made up with more granular scoping on dereferences, that may be a different consideration, but my view is that the language after this change has the semantics necessary to justify at the very least a majority of the optimizations that may ultimately be lost, and its up to rustc and llvm (and other compilers) to make use of those semantics if they wish to.

Just to follow up on some of the discussion, it wasn't immediately clear to me that types similar to Box, like Vec and Arc, genuinely don't have these semantics, even though it is implied by the fact that Box is unique in this regard. I think this is worth emphasising more in the text of the RFC, since in a very real sense, we've been going without these semantics for most parts of Rust totally fine, which further emphasises less will be lost by removing it for Box.

I mentioned Vec<T> in particular, and also mentioned std::unique_ptr<T> from C++, which is the most closely equivalent type, and lacks the same semantic implications (and also optimizations).

[scottmcm]

Just to follow up on some of the discussion, it wasn't immediately clear to me that types similar to Box, like Vec and Arc, genuinely don't have these semantics

And more than them just not having them, IIRC someone tried to implement Vec<T> as the obvious wrapper around Box<[MaybeUninit<T>]>, and in the process found out that lots of people are depending on Vec not having them.

[RalfJung]

would still love if there's more data showing the lack of returns on noalias optimisations, since it feels wrong that something with a lot of history and usage isn't helping that much

It helps for references. I suspect people added it for Box because "why not".

[theemathas]

I guess that the confusion here is how things should propagate. Like, the assumption that references are unique should extend directly from the assumption that the data they're referencing is unique. Otherwise, it feels like a disaster of UB?

It's currently unclear whether the existence of a &mut Box<T> even implies that there's an initialized non-null Box<T>. See rust-lang/unsafe-code-guidelines#412

[RalfJung]

Like, the assumption that references are unique should extend directly from the assumption that the data they're referencing is unique. Otherwise, it feels like a disaster of UB?

I don't understand either of these sentences, sorry.

&mut T references are unique because we say so, because we make the borrow checker and unsafe code enforce this, because it is part of their safety invariant.

[saethlin]

I believe noalias has been associated with Box since rust-lang/rust#11538. Note that the justification given at that time was

// `~` pointer parameters never alias because ownership is transferred

I think people are and have been reading way too much into the decision to put noalias on Box. That PR also added noalias to ~ returns, and that was probably unsound for 9 years until rust-lang/rust#106371. And justified with the same comment.

The vibes were good at the time, but we know so much more now.

[traviscross]

This proposed RFC kind of handwaves over what the opsem complications are that are motivating this. And for such a consequential RFC, the motivation section is rather short overall.

It'd be helpful for this motivating background context to be inlined and explored more deeply here, and for it to go into what the status of the various solutions to these problems might be.

[traviscross]

E.g., it'd be good to include the substance of @RalfJung's comment here somewhere:

What happened is that when applying the Stacked Borrows rules for Box to Vec, a bunch of existing code becomes UB. But that's more a statement about Stacked Borrows than about noalias. With Tree Borrows, things are already better, but even Tree Borrows has more UB than LLVM noalias. I am not aware of a single real-world usage of Vec that actually breaks the noalias rules.

[traviscross]

Suggested change

	- Alternative 2: Introduce a new type `AlisableBox<T>` which has the same interface as `Box<T>` but lacks the opsem implications that `Box<T>` has.
	- Alternative 2: Introduce a new type `AliasableBox<T>` which has the same interface as `Box<T>` but lacks the opsem implications that `Box<T>` has.

[traviscross]

Probably we should mention somewhere, in favor of adding a separate type like AliasableBox, that if we did that, we could then align other operations on that type to match in a way we can't do with Box. E.g., making a reference from an AliasableBox would be unsafe.

[nikomatsakis]

@RalfJung Your comment about vec surprised me a bit. I want to check my understanding. I think that the reason that Vec<T> has a more "low-level" API is basically people using Vec<u8> as a kind of convenient byte-buffer to implement things like arenas etc. It'd be helpful if you can point to some kind of summary of the sorts of issues encountered by stacked/tree borrows.

[clarfonthey]

Like, the assumption that references are unique should extend directly from the assumption that the data they're referencing is unique. Otherwise, it feels like a disaster of UB?

I don't understand either of these sentences, sorry.

&mut T references are unique because we say so, because we make the borrow checker and unsafe code enforce this, because it is part of their safety invariant.

Yeah, but if the pointers inside boxes aren't unique, then how can we guarantee that the mutable references are unique?

I'm sure I'm just misunderstanding what noalias means in these two different contexts, but what I'm wondering is, wouldn't aliasable boxes mean you could have two boxes pointing to the same memory, then unsoundly get a mutable reference to both at the same time?

[zachs18]

wouldn't aliasable boxes mean you could have two boxes pointing to the same memory, then unsoundly get a mutable reference to both at the same time?

If I understand correctly: Yes, having aliasing Boxes would let you get aliasing &muts in safe code (e.g. by dereferencing them and borrowing the result with &mut *bx). Therefore even under this RFC it sould still be a safety requirement (or library requirement) that Boxes not alias, i.e. you must not expose aliasing Boxes to arbitrary safe code, as opposed to the status quo where it is a validity requirement (or language requirement) that Boxes not alias, and is just immediate UB even if it only happens in localized unsafe code.

[chorman0773]

Indeed - it is not the intent of the RFC to require that arbitrary code remain valid in the presence of alised Box values, such a thing would be a breaking change - the library contract of Box isn't changed and still must be upheld. The proposal simply is to remove some UB from the language, when you do certain operations with Boxes in unsafe code.

Most collections act like this already, with Box being the outlier. It is possible, for example, to hold duplicate Vec<i32> values (with capacity > 0), but such values can't be fully used (including by dropping them implicitly or explicity) and can't be given to arbitrary code (safe or unsafe).

[RalfJung]

@RalfJung Your comment about vec surprised me a bit. I want to check my understanding. I think that the reason that Vec<T> has a more "low-level" API is basically people using Vec<u8> as a kind of convenient byte-buffer to implement things like arenas etc. It'd be helpful if you can point to some kind of summary of the sorts of issues encountered by stacked/tree borrows.

Yes to my knowledge the arena usecase is compatible with LLVM noalias semantics. I'm happy to discuss a concrete example if someone provides one and wonders how it interacts with noalias semantics, but don't have the time to produce a full writeup currently.

[RalfJung]

To give a simple example:

let mut v: Vec<i32> = Vec::with_capacity(128);
v.push_within_capacity(0).unwrap();
let elem0: &mut i32  = unsafe { &mut *v.as_mut_ptr().add(0) };
*elem0 = 12; // write to elem0 so that reference becomes "Active" in TB
let mut v = v; // retag v
v.push_within_capacity(1).unwrap();
// Carefully *not* using `v[1]` as that would create a slice conflicting with `elem0`.
let elem1: &mut i32  = unsafe { &mut *v.as_mut_ptr().add(1) };
mem::swap(elem0, elem1);

This is UB under Stacked Borrows and Tree Borrows semantics (if Vec is defined via Box<[_]>) because when v gets moved, there is a retag that conflicts with elem0.

However, LLVM noalias is fine with this program because noalias only exists on function arguments and is only active while the function runs. (In fact LLVM is even fine with using &mut v[1] instead of going via as_mut_ptr().)

[chorman0773]

So, elaborating on Allocators in the Motivation section got me thinking.
How does GlobalAlloc play in. Would GlobalAlloc not suffer similar issues if you wrote the allocator in Rust?

[traviscross]

We fixed this in RFC 3336, no? Either way, the text here should discuss that and the relationship of this to that.

cc @RalfJung

[chorman0773]

From what I read, that doesn't affect ManuallyDrop. It also only really needs to for Box, so this rfc would erase that use case entirely. MaybeDangling could be useful inside aggregates in the situation my footnote describes.

[traviscross]

From what I read, that doesn't affect ManuallyDrop.

It does though. It normatively states:

The ManuallyDrop<T> type internally wraps T in a MaybeDangling.

And we documented this in:

• Document subtleties of ManuallyDrop rust#130279

[traviscross]

So if this RFC erases that use case, it should either state normatively that it is updating RFC 3336 to not do that or explain why such wrapping is still needed.

[programmerjake]

just found an example where ManuallyDrop still needs MaybeDangling: https://rust-lang.zulipchat.com/#narrow/channel/213817-t-lang/topic/unsafe.20fields.20RFC/near/480108011

[chorman0773]

Frankly, I'd expect code like that should use MaybeDangling explicitly, on the field itself.

[RalfJung]

How does GlobalAlloc play in. Would GlobalAlloc not suffer similar issues if you wrote the allocator in Rust?

Global allocators involve some magic anyway that adds extra UB, so this could potentially be justified via that.

[RalfJung]

This looks to be a new guarantee? We currently don't normatively say anything about the layout of Box, AFAIK.

Also see this; Cc @CAD97

[chorman0773]

Stdlib docs promise it has the same layout and abi as a pointer, and also promises niche optimization.

https://doc.rust-lang.org/alloc/boxed/index.html#memory-layout

So long as T: Sized, a Box<T> is guaranteed to be represented as a single pointer and is also ABI-compatible with C pointers (i.e. the C type T*).

https://doc.rust-lang.org/core/primitive.fn.html#abi-compatibility

*const T, *mut T, &T, &mut T, Box<T> (specifically, only Box<T, Global>), and NonNull<T> are all ABI-compatible with each other for all T. They are also ABI-compatible with each other for different T if they have the same metadata type (<T as Pointee>::Metadata).

(The last sentence implies the abi, and thus layout, holds for T: !Sized pointers as well)

[RalfJung]

Ah okay :)

[ZuseZ4]

Just catching up with things. I wanted to share that for automatic differentiation (and likely GPU Kernels written in pure Rust) noalias is very relevant. At the llvm-dev meeting I've been presenting 5 benchmark that showed Performance Penalties in 4/5 Benchmarks ranging from 30% - 10x by removing noalias from the Rust code. Removing noalias from the C++ version of the benchmark also had large performance impacts (up to 4x), so it's not a pure Rust problem, and noalias is just generally very valuable for AutoDiff and GPU support. The only benchmark not suffering didn't had enough pointers/references for noalias to be valuable.

For now I have to admit that I have written all benchmarks using slices or arrays and not box, but both autodiff and GPU offload work on llvm-ir, and there are logical reasons about why noalias has such a big performance impact. So I don't see any reason why the penalty for noalias should be smaller for box. I will try to get at least some of these Benchmarks ported to use a Box instead of slices to prove my point, but for now I can share the existing slice/array versions that I used for my talk: EnzymeAD/Enzyme#1797

That being said, both autodiff and gpu-offloading are of course only experimental project goals, and noalias on box is of course less relevant than noalias on say slices. However, I think especially efficient GPU support is valuable beyond the scientific Computing/HPC/ML crowd and of interest for a lot of people. It might still be worth dropping it if it causes too many compiler issues, I haven't read up on the whole discussion yet. But we should at least be aware then that it likely will have relevant performance impact for these targets. If desired, I can also reach out to some of the LLVM people involved in GPU programming, they surely can provide more numbers/examples on noalias usage there.

Edit: Iirc Nikita also pointed out that people often pass Box behind a reference, which would keep noalias, so that might be enough for our cases (I will need to check the IR).

[GoldsteinE]

So I don't see any reason why the penalty for noalias should be smaller for box.

The idea is that box doesn’t get passed around that much: you usually just pass a reference, and mutable references have noalias.

(This is distinct from passing a box behind a reference / a reference to a box, which would have noalias on the reference, but not on the box)

[ZuseZ4]

Agree, but in our case the existence of 2+ Box types inside of a function (with at least one write to it) would often force us to cache the data behind all Boxes for autodiff (instead of just the one we modified), so it's not only about passing it to other functions. Vectors also don't have noalias iirc so I think also any combination of 2+ vec/box types existing in one function could then cause unnecessary copies.

I want to be clear that the average IR from Rust is still miles ahead of the average IR we get from C++ (since people rarely use restrict), but the difference would be that people manually could fix C++ by adding restrict, whereas Rust users wouldn't be able to add noalias to Rust's builtin types. It might still be worth to sacrifice performance in some niche cases for easier correctness, I'll try to write up in more detail under which cases lacking noalias would have an impact for the HPC/etc. community so people can decide if it's worth it. I just wanted to share this as an FYI, while I'm working out more details on it.

[chorman0773]

Within a function you already don't get noalias, due to limitations of llvm and what code rustc emits.

[ZuseZ4]

Are you talking about sret preventing noalias? Yes, we're following the work on the llvm side too, e.g. https://discourse.llvm.org/t/restrict-noalias-for-struct-members/78568, so we hope to increase the number of locations where llvm can use noalias info. I however don't have precise insights on how many of these llvm noalias improvements will later be useable by Rust and where we e.g. already allow unsafe code to alias and thus won't be able to add noalias.

[RalfJung]

Are you talking about sret preventing noalias?

No, he's talking about the fact that syntactically in LLVM IR noalias only exists for function arguments, not inside a function.

Agree, but in our case the existence of 2+ Box types inside of a function (with at least one write to it) would often force us to cache the data behind all Boxes for autodiff (instead of just the one we modified), so it's not only about passing it to other functions.

This can't be happening today since, as Connor said, noalias in LLVM is only for arguments. So are you speculating here about what benefits there might be in a hypothetical future where Box has noalias and also LLVM has a way to represent this inside a function, and Rust uses it?

Would be good to know which parts of your comments are based on concrete data and which are speculative.

[RalfJung]

and noalias is just generally very valuable for AutoDiff and GPU support

That's a good point that I had not considered. Would be good to get some data on the GPU side here, too.

[scottmcm]

I want to be clear that the average IR from Rust is still miles ahead of the average IR we get from C++ (since people rarely use restrict), but the difference would be that people manually could fix C++ by adding restrict, whereas Rust users wouldn't be able to add noalias to Rust's builtin types.

They could delegate to an inner function taking &mut to get noalias though, no?

(Like how autovectorization works better on &mut [T]/&[T] parameters than on Vec<T>, so you do the same thing there.)

[comex]

This can't be happening today since, as Connor said, noalias in LLVM is only for arguments. So are you speculating here about what benefits there might be in a hypothetical future where Box has noalias and also LLVM has a way to represent this inside a function, and Rust uses it?

Just to clarify, LLVM does have something called noalias that is not for arguments, namely noalias metadata. rustc will not directly produce this metadata, but it arguably could and should. Also note that this metadata can already end up in Rust code because LLVM converts function-argument-noalias to scoped-noalias upon inlining. (This does not alter the semantics.)

[RalfJung]

Yeah noalias metadata exists but it's largely a separate system from noalias arguments, as far as I can tell. If the semantics of noalias arguments are poorly understood, then the semantics of noalias metadata are a complete mystery -- I don't know anyone who has even the faintest clue of what the actual rules for the metadata are (in an operational sense).