Crater run with -Zrandomize-layout

[saethlin]

No description provided.

[saethlin]

@craterbot run mode=build-and-test start=stable end=stable+rustflags=-Zrandomize-layout

[craterbot]

👌 Experiment pr-132292 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[Noratrieb]

does crater make unstable flags on stable work?

[saethlin]

I bet no
@craterbot abort

[craterbot]

🗑️ Experiment pr-132292 deleted!

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[saethlin]

@craterbot run name=randomize-layout mode=build-and-test start=nightly-2024-10-28 end=nightly-2024-10-28+rustflags=-Zrandomize-layout

[craterbot]

👌 Experiment randomize-layout created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment randomize-layout is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment randomize-layout is completed!
📊 680 regressed and 106 fixed (531984 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[saethlin]

@craterbot run name=randomize-layout-1 mode=build-and-test start=nightly-2024-10-28 end=nightly-2024-10-28+rustflags=-Zrandomize-layout crates=https://crater-reports.s3.amazonaws.com/randomize-layout/retry-regressed-list.txt

[craterbot]

👌 Experiment randomize-layout-1 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment randomize-layout-1 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment randomize-layout-1 is completed!
📊 332 regressed and 11 fixed (1829 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[saethlin]

Experiment completed.

[scottmcm]

Well I clicked around randomly, and the first one looked legit, since "private hack" sounds exactly like someone misusing something:
https://crater-reports.s3.amazonaws.com/randomize-layout-1/nightly-2024-10-28%2Brustflags=-Zrandomize-layout/gh/DouglasDwyer.egui_wings/log.txt

[INFO] [stdout] error[E0080]: evaluation of constant value failed
[INFO] [stdout]   --> src/private_hack/mod.rs:50:21
[INFO] [stdout]    |
[INFO] [stdout] 50 |                     panic!("Style size not equal.");
[INFO] [stdout]    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the evaluated program panicked at 'Style size not equal.', src/private_hack/mod.rs:50:21
[INFO] [stdout]    |
[INFO] [stdout]    = note: this error originates in the macro `$crate::panic::panic_2021` which comes from the expansion of the macro `panic` (in Nightly builds, run with -Z macro-backtrace for more info)
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] error[E0512]: cannot transmute between types of different sizes, or dependently-sized types
[INFO] [stdout]    --> src/private_hack/mod.rs:142:13
[INFO] [stdout]     |
[INFO] [stdout] 142 |             transmute(value)
[INFO] [stdout]     |             ^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = note: source type: `private_hack::LabelSelectionState` (1600 bits)
[INFO] [stdout]     = note: target type: `egui::text_selection::LabelSelectionState` (1536 bits)

[scottmcm]

https://crater-reports.s3.amazonaws.com/randomize-layout-1/nightly-2024-10-28%2Brustflags=-Zrandomize-layout/gh/JoNil.rpi_gpu_game/log.txt

[INFO] [stdout] error[E0080]: evaluation of constant value failed
[INFO] [stdout]   --> src/main.rs:61:1
[INFO] [stdout]    |
[INFO] [stdout] 61 | const_assert!(Vertex; mem::size_of::<Vertex>() == 24);
[INFO] [stdout]    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ attempt to compute `0_usize - 1_usize`, which would overflow
[INFO] [stdout]    |
[INFO] [stdout]    = note: this error originates in the macro `_const_assert` which comes from the expansion of the macro `const_assert` (in Nightly builds, run with -Z macro-backtrace for more info)

https://crater-reports.s3.amazonaws.com/randomize-layout-1/nightly-2024-10-28%2Brustflags=-Zrandomize-layout/reg/reflexo-0.5.0-rc8/log.txt

[INFO] [stdout] error[E0080]: evaluation of constant value failed
[INFO] [stdout]    --> src/lib.rs:111:23
[INFO] [stdout]     |
[INFO] [stdout] 111 |         const _: () = assert!(core::mem::size_of::<ArchivedFontItem>() == 56);
[INFO] [stdout]     |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the evaluated program panicked at 'assertion failed: core::mem::size_of::<ArchivedFontItem>() == 56', src/lib.rs:111:23
[INFO] [stdout]     |
[INFO] [stdout]     = note: this error originates in the macro `assert` (in Nightly builds, run with -Z macro-backtrace for more info)

https://crater-reports.s3.amazonaws.com/randomize-layout-1/nightly-2024-10-28%2Brustflags=-Zrandomize-layout/reg/spacetimedb-query-planner-0.12.0/log.txt

[INFO] [stdout] error[E0308]: mismatched types
[INFO] [stdout]   --> src/logical/mod.rs:26:32
[INFO] [stdout]    |
[INFO] [stdout] 26 |         const _: [(); $size] = [(); ::core::mem::size_of::<$ty>()];
[INFO] [stdout]    |                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an array with a fixed size of 24 elements, found one with 32 elements
[INFO] [stdout]    |
[INFO] [stdout]   ::: src/logical/expr.rs:30:30
[INFO] [stdout]    |
[INFO] [stdout] 30 | static_assert_size!(RelExpr, 24);
[INFO] [stdout]    | --------------------------------
[INFO] [stdout]    | |                            |
[INFO] [stdout]    | |                            help: consider specifying the actual array length: `32`
[INFO] [stdout]    | in this macro invocation
[INFO] [stdout]    |
[INFO] [stdout]    = note: this error originates in the macro `static_assert_size` (in Nightly builds, run with -Z macro-backtrace for more info)

etc.

[saethlin]

Yep! The first task would be to filter out all the size assertions.

[the8472]

Some post reminded me that Foo<T> and Foo<U> currently use the same seed, so we're not even randomizing hard enough yet. I'll look into whether that can be fixed.

[scottmcm]

I wonder also if there's also a place here for a -Z randomize-layout=but-not-bigger or something, where for example an {a: String, b: u8, c: u8 } could be abc, acb, bca, or cba, but not bac or cab.

That way we could (mostly -- enums are complex) get past these size checks while still permuting layouts to detect stuff.

[the8472]

That should be fairly easy with the current logic, but the enum caveat is a pretty big one because it'd be more like a "but-mostly-not-much-bigger" 😅

[the8472]

PR for more randomization #133088

[the8472]

That PR has landed, might be worth another run.

[saethlin]

@craterbot run name=randomize-layout-2 mode=build-and-test start=nightly-2025-01-18 end=nightly-2025-01-18+rustflags=-Zrandomize-layout

[craterbot]

👌 Experiment randomize-layout-2 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment randomize-layout-2 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment randomize-layout-2 is completed!
📊 505 regressed and 136 fixed (570539 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[saethlin]

@craterbot run name=randomize-layout-3 mode=build-and-test start=nightly-2025-01-18 end=nightly-2025-01-18+rustflags=-Zrandomize-layout crates=https://crater-reports.s3.amazonaws.com/randomize-layout-2/retry-regressed-list.txt

[craterbot]

👌 Experiment randomize-layout-3 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment randomize-layout-3 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment randomize-layout-3 is completed!
📊 335 regressed and 15 fixed (3628 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[saethlin]

At a glance, numbers look the same as #132292 (comment) within noise.

[the8472]

Didn't that only rerun the ones that already regressed? I'd expect more regressions, not fewer.

[saethlin]

No. randomize-layout-2 is a fresh run, and randomize-layout-3 is a re-run of the recent regressions list.

Crater spuriously reports a lot of improvements and regressions, so unless you want to fish through all of its results by hand, re-running the regressions from a run is the only way to get a small report.