Subtree sync for rustc_codegen_cranelift #141557

bjorn3 · 2025-05-25T18:55:58Z

The main highlights this time are a Cranelift update and (thanks for beetrees) f16/f128 support.

@rustbot label +A-codegen +A-cranelift +T-compiler

…clif-2025-03-30

…Lapkin Rename `is_like_osx` to `is_like_darwin` Replace `is_like_osx` with `is_like_darwin`, which more closely describes reality (OS X is the pre-2016 name for macOS, and is by now quite outdated; Darwin is the overall name for the OS underlying Apple's macOS, iOS, etc.). ``@rustbot`` label O-apple r? compiler

…youxu Run coretests and alloctests with cg_clif in CI Part of rust-lang/rustc_codegen_cranelift#1290

- src\doc\nomicon\src\ffi.md should also have its ABI list updated

This will show a backtrace. Also added a reference to rust-lang/rustc_codegen_cranelift#171 in the unimplemented intrinsic error message.

…ntrinsic_error Replace trap_unimplemented calls with codegen_panic_nounwind

In preparation for future unwinding support. Part of rust-lang/rustc_codegen_cranelift#1567

Once writing the LSDA, it will need access to the Module to get a reference to the personality function and to define a data object for the LSDA. Part of rust-lang/rustc_codegen_cranelift#1567

It bugs me when variables of type `Ident` are called `name`. It leads to silly things like `name.name`. `Ident` variables should be called `ident`, and `name` should be used for variables of type `Symbol`. This commit improves things by by doing `s/name/ident/` on a bunch of `Ident` variables. Not all of them, but a decent chunk.

Remove the use of Rayon iterators This removes the use of Rayon iterators and the use of the `rustc-rayon` crate. `rustc-rayon-core` is still used however. In parallel loops, instead of a Rayon iterator a serial iterator are used to collect items into a `Vec` and we use a parallel loop over its elements using the new `par_slice` function which is built on `rustc-rayon-core`'s `join`. This change makes it easier to bring `rustc-rayon-core` in-tree. Tests using 7 threads: <table><tr><td rowspan="2">Benchmark</td><td colspan="1">Before</th><td colspan="2">After</th><td colspan="1">Before</th><td colspan="2">After</th><td colspan="1">Before</th><td colspan="2">After</th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th><td align="right">Physical Memory</td><td align="right">Physical Memory</td><td align="right">%</th><td align="right">Committed Memory</td><td align="right">Committed Memory</td><td align="right">%</th></tr><tr><td>🟣 clap:check</td><td align="right">0.4827s</td><td align="right">0.4828s</td><td align="right"> 0.02%</td><td align="right">201.23 MiB</td><td align="right">201.31 MiB</td><td align="right"> 0.04%</td><td align="right">279.03 MiB</td><td align="right">279.46 MiB</td><td align="right"> 0.15%</td></tr><tr><td>🟣 hyper:check</td><td align="right">0.1443s</td><td align="right">0.1401s</td><td align="right">💚 -2.91%</td><td align="right">126.42 MiB</td><td align="right">126.70 MiB</td><td align="right"> 0.22%</td><td align="right">199.79 MiB</td><td align="right">199.99 MiB</td><td align="right"> 0.10%</td></tr><tr><td>🟣 regex:check</td><td align="right">0.3252s</td><td align="right">0.3065s</td><td align="right">💚 -5.78%</td><td align="right">161.87 MiB</td><td align="right">161.78 MiB</td><td align="right"> -0.05%</td><td align="right">229.59 MiB</td><td align="right">230.23 MiB</td><td align="right"> 0.28%</td></tr><tr><td>🟣 syn:check</td><td align="right">0.5845s</td><td align="right">0.5876s</td><td align="right"> 0.53%</td><td align="right">197.01 MiB</td><td align="right">196.89 MiB</td><td align="right"> -0.06%</td><td align="right">267.62 MiB</td><td align="right">267.47 MiB</td><td align="right"> -0.06%</td></tr><tr><td>Total</td><td align="right">1.5367s</td><td align="right">1.5169s</td><td align="right">💚 -1.29%</td><td align="right">686.53 MiB</td><td align="right">686.68 MiB</td><td align="right"> 0.02%</td><td align="right">976.04 MiB</td><td align="right">977.14 MiB</td><td align="right"> 0.11%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9796s</td><td align="right">💚 -2.04%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> 0.04%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> 0.12%</td></tr></table> <table><tr><td rowspan="2">Benchmark</td><td colspan="1">Before</th><td colspan="2">After</th><td colspan="1">Before</th><td colspan="2">After</th><td colspan="1">Before</th><td colspan="2">After</th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th><td align="right">Physical Memory</td><td align="right">Physical Memory</td><td align="right">%</th><td align="right">Committed Memory</td><td align="right">Committed Memory</td><td align="right">%</th></tr><tr><td>🟠 clap:debug</td><td align="right">1.6371s</td><td align="right">1.6529s</td><td align="right"> 0.96%</td><td align="right">395.58 MiB</td><td align="right">396.21 MiB</td><td align="right"> 0.16%</td><td align="right">460.98 MiB</td><td align="right">461.52 MiB</td><td align="right"> 0.12%</td></tr><tr><td>🟠 hyper:debug</td><td align="right">0.3248s</td><td align="right">0.3210s</td><td align="right">💚 -1.16%</td><td align="right">155.16 MiB</td><td align="right">155.19 MiB</td><td align="right"> 0.02%</td><td align="right">219.21 MiB</td><td align="right">219.30 MiB</td><td align="right"> 0.04%</td></tr><tr><td>🟠 regex:debug</td><td align="right">1.0148s</td><td align="right">0.9929s</td><td align="right">💚 -2.16%</td><td align="right">297.96 MiB</td><td align="right">295.07 MiB</td><td align="right"> -0.97%</td><td align="right">354.53 MiB</td><td align="right">351.58 MiB</td><td align="right"> -0.83%</td></tr><tr><td>🟠 syn:debug</td><td align="right">1.3614s</td><td align="right">1.3717s</td><td align="right"> 0.76%</td><td align="right">319.10 MiB</td><td align="right">321.19 MiB</td><td align="right"> 0.65%</td><td align="right">378.90 MiB</td><td align="right">381.27 MiB</td><td align="right"> 0.62%</td></tr><tr><td>Total</td><td align="right">4.3381s</td><td align="right">4.3386s</td><td align="right"> 0.01%</td><td align="right">1.14 GiB</td><td align="right">1.14 GiB</td><td align="right"> -0.01%</td><td align="right">1.38 GiB</td><td align="right">1.38 GiB</td><td align="right"> 0.00%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9960s</td><td align="right"> -0.40%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> -0.03%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> -0.01%</td></tr></table>

Prepend temp files with per-invocation random string to avoid temp filename conflicts rust-lang#139407 uncovered a very subtle unsoundness with incremental codegen, failing compilation sessions (due to assembler errors), and the "prefer hard linking over copying files" strategy we use in the compiler for file management. Specifically, imagine we're building a single file 3 times, all with `-Csave-temps -Cincremental=...`. Let's call the object file we're building for the codegen unit for `main` "`XXX.o`" just for clarity since it's probably some gigantic hash name: ``` #[inline(never)] #[cfg(any(rpass1, rpass3))] fn a() -> i32 { 0 } #[cfg(any(cfail2))] fn a() -> i32 { 1 } fn main() { evil::evil(); assert_eq!(a(), 0); } mod evil { #[cfg(any(rpass1, rpass3))] pub fn evil() { unsafe { std::arch::asm!("/* */"); } } #[cfg(any(cfail2))] pub fn evil() { unsafe { std::arch::asm!("missing"); } } } ``` Session 1 (`rpass1`): * Type-check, borrow-check, etc. * Serialize the dep graph to the incremental working directory `.../s-...-working/`. * Codegen object file to a temp file `XXX.rcgu.o` which is spit out in the cwd. * Hard-link[^1] `XXX.rcgu.o` to the incremental working directory `.../s-...-working/XXX.o`. * Save-temps option means we don't delete `XXX.rgcu.o`. * Link the binary and stuff. * Finalize[^2] the working incremental session by renaming `.../s-...-working` to ` s-...-asjkdhsjakd` (some other finalized incr comp session dir name). Session 2 (`cfail2`): * Load artifacts from the previous *finalized* incremental session, namely the dep graph. * Type-check, borrow-check, etc. since the file has changed, so most dep graph nodes are red. * Serialize the dep graph to the incremental working directory `.../s-...-working/`. * Codegen object file to a temp file `XXX.rcgu.o`. **HERE IS THE PROBLEM**: The hard-link is still set up to point to the inode from `XXX.o` from the first session, so this also modifies the `XXX.o` in the previous finalized session directory. * Codegen emits an error b/c `missing` is not an instruction, so we abort before finalizing the incremental session. Specifically, this means that the *previous* session is the last finalized session. Session 3 (`rpass3`): * Load artifacts from the previous *finalized* incremental session, namely the dep graph. NOTE that this is from session 1. * All the dep graph nodes are green since we are basically replaying session 1. * codegen object file `XXX.o`, which is detected as *reused* from session 1 since dep nodes were green. That means we **reuse** `XXX.o` which had been dirtied from session 2. * Link the binary and stuff. This results in a binary which reuses some of the build artifacts from session 2, but thinks it's from session 1. At this point, I hope it's clear to see that the incremental results from session 1 were dirtied from session 2, but we reuse them as if session 1 was the previous (finalized) incremental session we ran. This is at best really buggy, and at worst **unsound**. This isn't limited to `-C save-temps`, since there are other combinations of flags that may keep around temporary files (hard linked) in the working directory (like `-C debuginfo=1 -C split-debuginfo=unpacked` on darwin, for example). --- This PR implements a fix which is to prepend temp filenames with a random string that is generated per invocation of rustc. This string is not *deterministic*, but temporary files are transient anyways, so I don't believe this is a problem. That means that temp files are now something like... `{crate-name}.{cgu}.{invocation_temp}.rcgu.o`, where `{invocation_temp}` is the new temporary string we generate per invocation of rustc. Fixes rust-lang#139407 [^1]: https://github.com/rust-lang/rust/blob/175dcc7773d65c1b1542c351392080f48c05799f/compiler/rustc_fs_util/src/lib.rs#L60 [^2]: https://github.com/rust-lang/rust/blob/175dcc7773d65c1b1542c351392080f48c05799f/compiler/rustc_incremental/src/persist/fs.rs#L1-L40

Add `f16`/`f128` support

…clif-2025-05-25

rustbot · 2025-05-25T18:56:05Z

⚠️ Warning ⚠️

There are issue links (such as #123) in the commit messages of the following commits.
Please remove them as they will spam the issue with references to the commit.
- 6424f0a
- ab514c9
- 9495eb5
- 0103c58
- 8e3d0b2
- 3ab6af0
The following commits have merge commits (commits with multiple parents) in your changes. We have a no merge policy so these commits will need to be removed for this pull request to be merged.
- 0700696
- 0da0dac
- 23f12ff
- 322bba0
- 3816385
- 41dbfa7
- 43f4232
- 44bbe63
- 4a49ff0
- 62c72fc
- 6412cfb
- 6b06289
- 7b670d2
- 88c48cd
- 91c9660
- 934931b
- aa04a27
- cd8a1ad
- d7a6a71
- d9dac3c
- e864f0f
- f55c2c0
You can start a rebase with the following commands:
```
$ # rebase
$ git pull --rebase https://github.com/rust-lang/rust.git master
$ git push --force-with-lease
```

rustbot · 2025-05-25T18:58:29Z

The list of allowed third-party dependencies may have been modified! You must ensure that any new dependencies have compatible licenses before merging.

cc @davidtwco, @wesleywiser

bjorn3 · 2025-05-25T18:58:56Z

@bors r+ p=1 subtree sync

bors · 2025-05-25T18:58:58Z

📌 Commit 4aed799 has been approved by bjorn3

It is now in the queue for this repository.

bors · 2025-05-25T21:06:28Z

⌛ Testing commit 4aed799 with merge 9f8929f...

bors · 2025-05-26T00:17:30Z

☀️ Test successful - checks-actions
Approved by: bjorn3
Pushing 9f8929f to master...

github-actions · 2025-05-26T00:20:47Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 283db70 (parent) -> 9f8929f (this PR)

Test differences

No test diffs found

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 9f8929fbeca4b5c2302b326606ae800156915840 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

aarch64-gnu-debug: 3950.7s -> 4935.0s (24.9%)
dist-armv7-linux: 5313.3s -> 6550.8s (23.3%)
x86_64-apple-1: 6342.2s -> 7805.2s (23.1%)
aarch64-gnu: 6562.6s -> 8013.0s (22.1%)
x86_64-apple-2: 4195.3s -> 4918.9s (17.2%)
aarch64-apple: 4309.1s -> 3989.5s (-7.4%)
x86_64-msvc-ext3: 7957.2s -> 7547.8s (-5.1%)
dist-various-2: 3345.2s -> 3189.1s (-4.7%)
x86_64-gnu: 6510.6s -> 6811.9s (4.6%)
dist-loongarch64-musl: 5617.9s -> 5370.7s (-4.4%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-05-26T03:13:12Z

Finished benchmarking commit (9f8929f): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary 9.5%, secondary 2.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	9.5%	[9.1%, 9.8%]	2
Regressions ❌ (secondary)	2.2%	[2.2%, 2.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	9.5%	[9.1%, 9.8%]	2

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 775.708s -> 776.208s (0.06%)
Artifact size: 366.25 MiB -> 366.34 MiB (0.03%)

madsmtm and others added 30 commits March 25, 2025 21:53

Rename is_like_osx to is_like_darwin

c56c2b7

Merge commit 'ba315abda789c9f59f2100102232bddb30b0d3d3' into sync_cg_…

15dbafa

…clif-2025-03-30

Merge branch 'sync_from_rust'

4a49ff0

Allow formatting example/gen_block_iterate.rs

625b800

Unset RUSTC_WRAPPER in cg_clif's build system

e58dd25

Run coretests and alloctests with cg_clif in CI

bb2b3d0

Fix testing with randomized layouts enabled

e3a8d9c

Sync from rust 00095b3

43f4232

Rustup to rustc 1.88.0-nightly (00095b3 2025-04-03)

4807c29

Fix rustc test suite

b0c23f7

Tell rustfmt to use the 2024 edition

829413d

Auto merge of rust-lang#139213 - bjorn3:cg_clif_test_coretests, r=jie…

0700696

…youxu Run coretests and alloctests with cg_clif in CI Part of rust-lang/rustc_codegen_cranelift#1290

update docs

4a8026c

- src\doc\nomicon\src\ffi.md should also have its ABI list updated

Sync from rust 2fa8b11

6b06289

Rustup to rustc 1.88.0-nightly (2fa8b11 2025-04-06)

25f263d

Preserve rustc_literal_escaper with --sysroot llvm

0e9a854

Simplify temp path creation a bit

ab84fe6

Prepend temp files with a string per invocation of rustc

68dd8b3

Sync from rust e643f59

44bbe63

Rustup to rustc 1.88.0-nightly (e643f59 2025-04-07)

b69a478

Replace trap_unimplemented calls with codegen_panic_nounwind

6424f0a

This will show a backtrace. Also added a reference to rust-lang/rustc_codegen_cranelift#171 in the unimplemented intrinsic error message.

Merge pull request rust-lang#1568 from rust-lang/better_unsupported_i…

91c9660

…ntrinsic_error Replace trap_unimplemented calls with codegen_panic_nounwind

Reduce visibility of a couple of functions

420e44f

Pass UnwindAction to a couple of functions

ab514c9

In preparation for future unwinding support. Part of rust-lang/rustc_codegen_cranelift#1567

Pass Module to UnwindContext

9495eb5

Once writing the LSDA, it will need access to the Module to get a reference to the personality function and to define a data object for the LSDA. Part of rust-lang/rustc_codegen_cranelift#1567

Remove the use of Rayon iterators

180bc6c

beetrees and others added 5 commits May 24, 2025 13:51

Enable tests and compiler-builtins for f16/f128

02195f5

Merge pull request rust-lang#1574 from beetrees/f16-f128-mvp

0da0dac

Add `f16`/`f128` support

Sync from rust 5e16c66

aa04a27

Rustup to rustc 1.89.0-nightly (5e16c66 2025-05-24)

979dcf8

Merge commit '979dcf8e2f213e4f4b645cb62e7fe9f4f2c0c785' into sync_cg_…

3816385

…clif-2025-05-25

rustbot added has-merge-commits PR has merge commits, merge with caution. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 25, 2025

Update tidy exceptions

4aed799

rustbot added A-tidy Area: The tidy tool T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 25, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label May 26, 2025

bors merged commit 9f8929f into rust-lang:master May 26, 2025
7 checks passed

rustbot added this to the 1.89.0 milestone May 26, 2025

This was referenced May 26, 2025

Add lint against (some) interior mutable consts #132146

Open

lint ImproperCTypes: overhaul (take 2 of "better handling of indirections") #134697

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Subtree sync for rustc_codegen_cranelift #141557

Subtree sync for rustc_codegen_cranelift #141557

bjorn3 commented May 25, 2025

Uh oh!

rustbot commented May 25, 2025

Uh oh!

rustbot commented May 25, 2025

Uh oh!

bjorn3 commented May 25, 2025

Uh oh!

bors commented May 25, 2025

Uh oh!

bors commented May 25, 2025

Uh oh!

bors commented May 26, 2025

Uh oh!

Uh oh!

github-actions bot commented May 26, 2025

Uh oh!

rust-timer commented May 26, 2025

Uh oh!

Uh oh!

Subtree sync for rustc_codegen_cranelift #141557

Subtree sync for rustc_codegen_cranelift #141557

Conversation

bjorn3 commented May 25, 2025

Uh oh!

rustbot commented May 25, 2025

Uh oh!

rustbot commented May 25, 2025

Uh oh!

bjorn3 commented May 25, 2025

Uh oh!

bors commented May 25, 2025

Uh oh!

bors commented May 25, 2025

Uh oh!

bors commented May 26, 2025

Uh oh!

Uh oh!

github-actions bot commented May 26, 2025

Test differences

Job duration changes

Uh oh!

rust-timer commented May 26, 2025

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

Uh oh!