Reduce our MSRV as much as possible

[joshlf]

Our MSRV is currently 1.61, which is higher than the MSRVs of many crates. Here are MSRVs of the subset of the top 100 most-recently-downloaded crates that contain unsafe code:

Top 100 crates

Crate	MSRV	Unsafe
syn	1.56	search
bitflags	1.56	search
hashbrown	1.63	search
regex-syntax	1.65	search
proc-macro2	1.56	search
quote	1.56	search
base64	1.48	search
libc	1.13	search
unicode-ident	1.31	search
serde	1.31	search
cfg-if	unknown	search
rustix	1.63	search
rand_core	unknown	search
serde_derive	1.56	search
aho-corasick	1.60	search
indexmap	1.63	search
itoa	1.36	search
regex-automata	1.65	search
rand	unknown	search
getrandom	1.36	search

We should work to reduce our MSRV as much as possible so that more crates can replace unsafe code with zerocopy.

Mentoring instructions

• Repeat the following steps as many times as possible:
  • Determine what is preventing our MSRV from being one version earlier
  • Determine whether there's a workaround for the same behavior that is compatible with an earlier MSRV

Conditional compilation problems

One problem with lowering our MSRV is that some features we want to use were introduced in more recent Rust versions. Naively, one might expect that this makes lowering our MSRV dead in the water. However, it's possible to work around this, albeit with a lot of machinery.

The core insight is that it's okay for zerocopy to provide certain features only when compiled with certain Rust versions. For example, imagine that a certain feature is only available on 1.60 and above. Downstream crates can be in one of two buckets:

• Their MSRV is 1.60 or above. They will never compile with Rust <= 1.60, and so they will never compile zerocopy without support for this feature.
• Their MSRV is below 1.60. They (hopefully) test their crate with their MSRV, and so are unable to publish code which makes use of this feature (unless they use their own conditional compilation logic, which just kicks the can to their downstream crates).

In both cases, it's consistent with semver rules for us to enable certain features only on more recent Rust versions.

This brings us to our first requirement: We need to support detecting the Rust toolchain version at compile time.

---

If we take this approach, it introduces a new problem: now zerocopy's behavior can differ by toolchain. If we continue to only test on MSRV, stable, and nightly toolchains, then there will be some toolchain versions which introduce different behavior relative to their parent, but on which we don't test. For example, if our MSRV is 1.56, and our pinned stable toolchain is 1.70, but we have a feature that is enabled on versions after 1.65, then it's possible that that feature is buggy on 1.65, 1.66, 1.67, 1.68, and 1.69. This implies that we also need to test on 1.65 (the first Rust version for which the feature is enabled). We need to do this for any Rust version for which we enable new behavior.

This brings us to our second requirement: In CI, we need to test on every Rust version which introduces new behavior.

---

This introduces another problem, though. How do we actually implement this? Naively, we could write our build.rs script to perform toolchain detection based on hard-coded toolchain versions, and then separately update our CI configuration to test on these toolchain versions. However, this risks bit rot: we could easily change the hard-coded versions in either place, but forget to update them in the other place. There would be no automatic way to detect this.

Instead, we should have one source of truth for the list of toolchain versions. This brings us to our third requirement: Both build.rs and our CI scripts should parse a single source of truth for the list of toolchain versions.

---

All three of these requirements are implemented in #843.

[polarathene]

I'm a little curious about the impact of this (as a general goal, not specifically the impact to zerocopy).

• As you raise PRs to iteratively lower the MSRV, you need to make more sacrifices that affects the maintenance of zerocopy?
• And then eventually raising MSRV again to undo these changes?

Perhaps it's more feasible for zerocopy due to minimal dependencies, so most of the concerns below can be disregarded 😅

---

In #557 you reference libc MSRV and targets of 1.30 (8% compatible) and 1.31 (19% compatible), with MSRV compatibility of current crates.io ecosystem:

Your current target reached (1.57) is 59% compatible:

Once you pass 1.56.0 you've lost support for depending on any crates using edition = 2021, while some crates for example require 1.60 due to usage of new Cargo.toml syntax introduced then but those releases are completely invisible/ignored by the resolver on earlier toolchains it seems. The rust-version field was also introduced in 1.56.0 (which I see zerocopy leverages).

In contrast of that full ecosystem, the lib.rs stats link also shows the MSRV compatibility of more recently updated crates:

Note the widening "broken deps" compatibility, along with the steep "incompatible" increase after going below edition = "2021" compatibility with 1.55.

Possibly relevant: mitsuhiko/insta#289 (comment)

syn bumped its MSRV up to 1.56 with the v2 release. This alone will bump the MSRV of a lot of projects to at least 1.56 once updates propagate out

I don't think we depend on syn with all features off, but any feature requiring serde will pull it in with a non-semver breaking change

---

Until the MSRV resolver work is stabilized and crates adopt declaring rust-version to better communicate compatibility through the dependency chain, the further back in toolchain releases you go the more problems you run into resolving indirect dependencies.

What once resolved and installed correctly via Cargo.toml fails on the aging toolchain as time marches forward (rust-version does not avoid that issue unless all dependencies adopt it). Managing a Cargo.lock becomes a point of friction, especially if starting a new project with toolchains from 8 years ago (assuming the environment to support requires it).

---

What will your plan be once you reached the MSRV target you're aiming for?

Will you continue to support that for N years? Semver bumps are apparently advised to not be affected by MSRV bumps for valid reasons, so using minor version bumps for MSRV and keeping those release streams updated with patch releases doesn't appear to be advised? (crates like tokio appear to do this, but due to major version of 1 instead of 0, that is viable AFAIK)

While the MSRV policy resolver work should improve on this situation, by the time crates have more widely adopted declaring rust-version it's not likely to have a baseline that satisfies very old toolchain releases.

• When the resolver isn't able to identify a compatible rust-version, the last release without one that matches the semver range is selected but likely won't be too far from the last known rust-version declared and thus fail.
• Take hashbrown 0.14.2 for example. It declares rust-version = "1.63.0", but will require 1.64.0` by default.
  • You need to opt-out of a feature or Cargo.lock pin the allocator-api2 dependency to a compatible version.
  • Even if that dependency later adopts a rust-version, the earlier releases without it would still resolve for the older toolchains.
  • Likewise that dependency can satisfy an MSRV of 1.56.0 by minimizing it's own dependency on once_cell to the lower semver range 0.15.0, but that sets a hard limit due to edition = "2021" usage.
• Earlier hashbrown releases had rust-version = "1.56.0" but again due to a dependency release after hashbrown releases, the compatible crates in semver had been yanked (ahash), while newer point releases implicitly bumped MSRV to 1.60.0 due to Cargo.toml syntax change that older tool chain could not parse so nothing was resolvable for earlier toolchains. Depending on when you installed / resolved the ordered-multimap = "0.4" package semver, you might have had a newer release, or today you'd be reverted back to 0.4.0 (unless attempting to use update -Z msrv-policy, which resolves newer crates but fails to install).

---

You'd have a better idea of all this I assume, along with actual users requiring it? ( #557 is marked with the label customer-request)

With Rust 1.31 (introduces edition = "2018"), and LTS distros from that year using that or older, some with LTS support windows of 10 years, I assume that could avoid a bump until around 2028?

• Does zerocopy stop maintaining MSRV compatibility with releases before then?
• Is the goal to just reach a low MSRV, and then bring up the MSRV again with rust-version usage?
• Or adopt a 1.0.0 major release to introduce MSRV bumps via minor releases and allow patch releases to those older MSRV minors similar to tokio?
• Some other plan to maintain MSRV compatibility with support for backports of fixes?

[joshlf]

Thanks for the detailed analysis! I think I can answer most of your questions, but let me know if I've missed anything:

• Our current MSRV policy is just that we consider an MSRV bump to be a semver-breaking change. We don't promise anything about when we'll make such changes, other than promising that they'll only happen in semver-breaking releases (e.g., 0.7.X -> 0.8.0).
  • For users who are relying on zerocopy but not exposing it in their API, we expect them to be able to stay on the old version train if the new MSRV is problematic, or to upgrade if they are compatible with the new MSRV.
  • For users who are relying on zerocopy and exposing it in their API, they already need to handle other (non-MSRV-related) breaking changes, e.g. by providing one feature per zerocopy version, and MSRV would not introduce a new concern that didn't already exist with other API changes introduced in the new version.
• The libc issue refers to increasing libc's MSRV to 1.31, but we don't intend to decrease our MSRV to 1.31. Instead, the thinking there is that increasing libc's MSRV to 1.31 would make it easier to support an optional zerocopy feature. That feature would require a higher MSRV than 1.31 (namely, whatever zerocopy's MSRV is).
• We don't have any concrete plans to reach 1.0 (we intend to eventually, but we aren't currently planning for it concretely), and we don't have any particular thoughts on whether or how our MSRV policy will change at that point.

[joshlf]

0.8's MSRV is 1.56. That has taken a lot of engineering effort, and we continue to maintain infrastructure and complexity required to support that MSRV. As a result, we no longer plan to continue to lower our MSRV. 0.8 will be published with an MSRV of 1.56, and future version trains (0.9 and beyond) will likely have significantly higher MSRVs.