Document not found (404)
+This URL is invalid, sorry. Please use the navigation bar or search to continue.
+ +This URL is invalid, sorry. Please use the navigation bar or search to continue.
+ +Each commit also generates what are called “alt” builds. +These are builds of rustc with some different options set. +As of August 2023, these include:
+rust.parallel-compilerllvm.assertionsrust.verify-llvm-irFor more information on these settings, see the config.toml docs.
+These alt settings are defined in ci/run.sh.
Alt builds are only available for a few targets.
+Look for the -alt builds in ci.yml.
This can be useful if you are bisecting an LLVM issue. +With LLVM assertions enabled, alt builds have checks that can help identify broken assumptions.
+Alt builds are only made for commit builds, and not nightly releases.
+You will need to specify --by-commit (or use a hash in the --start or --end flags) to only use commit builds.
cargo bisect-rustc --alt --by-commit
+cargo-bisect-rustc does a binary search for the regression using a start and end boundary.
+You can specify these boundaries with the --start and --end CLI flags.
+There are several ways to specify what those boundaries are.
+If you run the command without specifying the boundaries, it will search for them automatically:
# No --start or --end flags
+cargo bisect-rustc
+This will assume the latest nightly is a regression (the end boundary). +It will then search backwards until it can find a nightly that passes to use as the start boundary. +Bisection can usually go faster if you happen to know the start boundary, so that it doesn’t need to search for it.
+--start and --end are optional.
+If --start is not specified, then it will try to find the start range automatically.
+If --end is not specified, it will assume it is the most recently available.
You can pass a date in the form YYYY-MM-DD to the --start and --end flags.
+It will download the nightly corresponding to that date, and then begin bisecting those nightlies.
cargo bisect-rustc --start=2018-08-14 --end=2018-10-11
+If the nightly with the regression was within the past 167 days, then it will automatically start bisecting the individual PRs merged on that day using Git commit boundaries.
+You can pass the particular git commit hash of a PR as a boundary. +The Rust project keeps the builds of every merged PR for the last 167 days. +If you happen to know the PR to use as a boundary, you can pass the SHA-1 hash of that PR.
+cargo bisect-rustc \
+ --start=6323d9a45bdf0ac2a9319a6a558537e0a7e6abd1 \
+ --end=866a713258915e6cbb212d135f751a6a8c9e1c0a
+There are several ways to determine the SHA-1 hash for a PR.
+rust-lang:master”.
+You can copy that hash to use as a boundary.
+If the PR was merged as part of a rollup, you will need to use the hash of the rollup instead.
+You’ll need to look through the PR messages to see if the PR was mentioned from a rollup PR.git log --first-parent upstream/master (where upstream is your origin name for rust-lang/rust).
+This will show all the top-level commits.
+You can then search for your PR.++Note: If the PR was merged after the most recent nightly, you’ll need to be sure to also specify the
+--endrange. +Otherwise it will assume the most recent nightly is the end and it won’t work if the start is after the end.
If the regression is found in a rollup PR, then cargo-bisect-rustc will bisect the individual PRs within the rollup.
+This final bisection is only available for x86_64-unknown-linux-gnu since it is using the builds made for the rustc performance tracker.
++Note: If you specify date boundaries, then you can use the
+--by-commitCLI option to force it to use PR commits instead of nightlies.
The boundary can be specified with a git release tag.
+This is useful if you know something works in one release and not another, but you don’t happen to know which nightly this corresponds with.
+When given a tag, cargo-bisect-rustc will try to find the nightly that corresponds with that release.
+For example:
cargo bisect-rustc --start=1.58.0 --end=1.59.0
+When writing your test and picking a bisection range, you should be careful to ensure that the test won’t vary between pass/fail over the bisection range. +It should only transition from good to bad once in the bisection range (it must change +monotonically).
+In the following example, cargo-bisect-rustc will find one of the transitions, but that may not be the true root cause of the issue you are investigating.
nightly-2023-02-01 baseline **start**
+nightly-2023-02-02 baseline
+nightly-2023-02-03 baseline
+nightly-2023-02-04 regression
+nightly-2023-02-05 regression
+nightly-2023-02-06 baseline
+nightly-2023-02-07 regression
+nightly-2023-02-08 regression **end**
+Here it may either find 2023-02-04 or 2023-02-07 as the regression.
+The following are some suggestions for avoiding or dealing with this problem:
+-vv flag (very verbose) to display the output from the compiler to make sure it is what you expect.--prompt flag to inspect the output and verify each step.cargo-bisect-rustc can be used to check for Clippy regressions, too.
+You’ll need to instruct it to download clippy, and run the command correctly:
cargo bisect-rustc --start=1.67.0 --end=1.68.0 -c clippy -- clippy
+Note that depending on what you are looking for, this may just find a PR that syncs the rust-clippy repo to rust-lang/rust.
+You may be able to scan the list of changes in that PR to discover what you are looking for.
+If the list of changes is too big or nothing is jumping out as a possible culprit, then consider using git bisect on the clippy repo itself (which will require building clippy).
By default, cargo-bisect-rustc only fetches rustc, cargo, rustdoc, and the standard library for the host.
+You may need additional Rustup Components to run your test.
+Some examples of when this might be needed are:
x86_64-pc-windows-gnu host may need the rust-mingw component.If you are testing cross-compilation, use the --target option to download the standard library for the target you are using.
The following example shows how to use components to do a bisection with Cargo’s build-std feature.
+cargo-bisect-rustc --start=2022-11-01 --end=2022-11-20 -c rust-src -- build -Zbuild-std
+++ +Note: The
+--with-srcoption is an alias for-c rust-src.
+The--with-devoption is an alias for-c rustc-dev -c llvm-tools.
The following is an example of checking when the diagnostic output of rustc changes.
+For example, this can check when either the wording has changed, or a different error or warning is produced.
#109067 is an example of where this is necessary.
+A warning started being emitted, and it is the kind of warning that cannot be turned into an error with deny(warnings).
The following script is intended to be used with the --script option (set the executable flag on the script, chmod u+x):
#!/bin/sh
+
+OUTPUT=`cargo check 2>&1`
+# Comment out this test if your example is intended to fail.
+if [ $? -ne 0 ]
+then
+ echo "Build unexpectedly failed: $OUTPUT"
+ exit 1
+fi
+# Display the output for debugging purposes.
+# Run `cargo-bisect-rustc` with `-vv` to view the output.
+echo "$OUTPUT"
+# This indicates a regression when the text "non-ASCII" is in the output.
+#
+# If the regression is when the text is *not* in the output, remove the `!` prefix
+# (and customize the `--term-old` and `--term-new` CLI options if you want).
+! echo "$OUTPUT" | grep "non-ASCII"
+Then run something like:
+cargo bisect-rustc --start=1.67.0 --end=1.68.0 --script ./test.sh \
+ --term-old="No warning" --term-new="Found non-ASCII warning"
+cargo-bisect-rustc can be used to scan for changes in the documentation shipped with each release.
+This includes all the books and standard library documentation.
+To do this, instruct it to download the component, and use a script that scans for whatever you are looking for.
+You can use rustup doc --path or rustc --print=sysroot to find the proper location.
+For example:
test.sh:
#!/bin/sh
+
+# Exit if any command fails.
+set -e
+
+STD=`dirname $(rustup doc --std --path)`
+
+# Checks if a particular file exists.
+# This could also be `grep` or any other kinds of tests you need.
+if [ -e $STD/io/error/type.RawOsError.html ]
+then
+ echo "found"
+ exit 1
+fi
+And run with:
+cargo bisect-rustc --start 1.68.0 --end 1.69.0 -c rust-docs --script ./test.sh \
+ --term-old="Did not find" --term-new="Found"
+++ +Note: This may not work on all targets since
+cargo-bisect-rustcdoesn’t properly handle rustup manifests, which alias some targets to other targets. +Use--host x86_64-unknown-linux-gnuin that situation.
Some tests may fail randomly.
+The following script is an example that will run rustc repeatedly to check for a failure.
+This example is from #108216 (which requires macOS).
test.sh:
#!/bin/sh
+
+rm -rf *.o incremental foo
+
+echo "fn main() { let a: i64 = 1 << 64; }" > foo1.rs
+echo "fn main() { let a: i64 = 1 << 63; }" > foo2.rs
+
+ARGS="--crate-name foo -C split-debuginfo=unpacked -C debuginfo=2 -C incremental=incremental"
+
+for i in {1..20}
+do
+ echo run $i
+ rustc foo1.rs $ARGS && { echo "ERROR: first build should have failed"; exit 1; }
+ rustc foo2.rs $ARGS || { echo "ERROR: second build should have passed"; exit 1; }
+ ./foo || { echo "ERROR: executing should have passed"; exit 1; }
+done
+This test can be run with:
+cargo bisect-rustc --start=1.57.0 --end=1.58.0 --script=./test.sh
+In general, configure the script to perform whichever actions you need in a for loop that runs enough times that you have a high confidence it has found the regression.
Testing for regressions with incremental compilation may require running a command multiple times. +The following illustrates an example for #87384 which only generates a warning the second time a build is run with incremental. +Previously no warning was emitted.
+foo.rs:
+#![type_length_limit = "95595489"] + +pub fn main() { + println!("Hello, world!"); +}
Create a script test.sh:
#!/bin/sh
+
+# Exit if any command fails.
+set -e
+
+rm -rf incremental
+rustc foo.rs --crate-type lib -C incremental=incremental
+echo second
+OUTPUT=`rustc foo.rs --crate-type lib -C incremental=incremental 2>&1`
+echo $OUTPUT
+! echo "$OUTPUT" | grep \
+ "crate-level attribute should be in the root module"
+Run this script with:
+cargo-bisect-rustc --start 1.54.0 --end 1.55.0 --script ./test.sh
+The following chapters show examples of different ways of using cargo-bisect-rustc.
You may want to reuse the toolchains downloaded by cargo-bisect-rustc for doing further analysis or debugging.
+Or, while setting up your regression test, you may need to adjust your test and script several times, and downloading the same toolchains multiple times can be quite slow.
You can do this with the --preserve option.
cargo bisect-rustc --start=2023-01-01 --end=2023-02-01 --preserve
+The toolchains will be kept in your Rustup home directory (typically ~/.rustup/toolchains).
Toolchains for nightlies will have the form of bisector-nightly-YYYY-MM-DD-<target>.
+Toolchains for PR artifacts will have the form of bisector-ci-<hash>-<target>.
You can run these toolchains using a Rustup override, like this:
+cargo +bisector-nightly-2023-03-18-x86_64-unknown-linux-gnu build
+# or...
+cargo +bisector-ci-e187f8871e3d553181c9d2d4ac111197a139ca0d-x86_64-unknown-linux-gnu build
+When you are done, you’ll probably want to clean up these directories since they use a lot of space. +The easiest method is to just delete the directories:
+rm -rf ~/.rustup/toolchains/bisector-*
+The --install option can be used to only install a toolchain.
+This won’t do a bisection, it is just for fetching a toolchain for testing.
cargo bisect-rustc --install e187f8871e3d553181c9d2d4ac111197a139ca0d
+++ +Note: See also
+rustup-toolchain-install-masterwhich is specialized for installing CI artifacts.
cargo-bisect-rustc can be used to check for Rustdoc regressions, too.
+All you need to do is instruct it to use the correct command.
The following example will check to find a regression when cargo doc suddenly starts to fail.
cargo bisect-rustc --start=2022-08-05 --end=2022-09-09 -- doc
+Some rustdoc regressions might be in the generated HTML output. +To scan the output, you can use a script like the following:
+test.sh:
#!/bin/sh
+
+# Exit if any command fails.
+set -e
+
+cargo doc
+
+grep "some example text" $CARGO_TARGET_DIR/doc/mycrate/fn.foo.html
+This can be used with the --script option:
cargo-bisect-rustc --start=2023-01-22 --end=2023-03-18 --script=./test.sh \
+ --term-old="Found example text" --term-new="Failed, or did not find text"
+Some regressions may involve the compiler hanging or taking an unusually long time to run.
+The --timeout CLI option can be used to check for this.
+Let’s use #89524 as an example.
+A particular combination of factors caused the compiler to start to hang.
Change Cargo.toml to the following:
[package]
+name = "slow"
+version = "0.1.0"
+
+[dependencies]
+config = "=0.9.3"
+
+[profile.release]
+panic = "abort"
+codegen-units = 1
+Then use the timeout option:
+cargo-bisect-rustc --start=2021-09-01 --end=2021-10-02 --timeout 30 -- build --release
+You may need to adjust the timeout value based on the speed of your system.
+++ +Note:
+--timeoutis currently not working on macOS. See https://github.com/rust-lang/cargo-bisect-rustc/issues/232.
Using the --script option on Windows can be cumbersome because Windows does not support #! scripts like Unix does, and the built-in scripting can also be awkward.
+The following sections show the different ways you can use scripting.
You can use DOS-style .bat files:
test.bat:
(cargo check 2>&1) | find "E0642"
+This can be executed directly with:
+cargo-bisect-rustc --script ./test.bat
+But .bat can be challenging to do more complex options, or you may not be familiar with it.
You can’t execute .ps1 Powershell files directly, so you will need to use pwsh to launch them:
test.ps1:
( cargo check 2>&1 ) | grep E0642
+if ( -Not $? ) {
+ exit 1
+}
+This can be run with:
+cargo-bisect-rustc --script pwsh -- -File ./test.ps1
+If you have Git-for-Windows installed, then you can use its copy of bash to run bash scripts:
+test.sh:
#!/bin/bash
+
+cargo check 2>&1 | grep E0642
+This can be run with:
+cargo-bisect-rustc --script "C:\\Program Files\\Git\\usr\\bin\\bash.exe" -- ./test.sh
+This also works if you have bash from something like msys2 installed.
+ +Some bisections don’t require Cargo.
+You can use the --without-cargo option to skip installing cargo which can speed up the bisection since it doesn’t need to download cargo, and doesn’t have the overhead of running cargo.
+You will need to pair this with --script since cargo-bisect-rustc assumes projects use Cargo.
For example, using a simple rustc command:
cargo-bisect-rustc --start=2022-11-01 --end=2022-11-20 --without-cargo --script=rustc -- foo.rs
+++ +Note: You can use
+--without-cargowhile still using a Cargo project. +Rustup will fall back to usingcargofrom your installed nightly, beta, or stable toolchain. +However, this isn’t recommended sincecargois only intended to work with the version it is released with, and can sometimes be incompatible with different versions. +But if you are bisecting a very recent change, then you can probably get away with it.
There are some rare cases where you may need to build rustc with custom options, or otherwise work around issues with pre-built compilers not being available.
+For this you can use git bisect to build the compiler locally.
It can be helpful to use the --first-parent option so that it only bisects the merge commits directly reachable on the master branch.
+Otherwise the bisecting may land on intermediate commits from within a PR which may not build or test correctly.
To start the bisection, specifying the boundaries where the bisection will start:
+git bisect start --first-parent
+git bisect good 96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef
+git bisect bad a00f8ba7fcac1b27341679c51bf5a3271fa82df3
+Then, build the compiler as needed and run your tests to check for a regression:
+./x.py build std
+rustc +stage1 foo.rs
+You may want to consider running ./x.py clean if you are running into issues since changes to the internal structures of build artifacts aren’t always versioned, and those changes can be incompatible.
+Incremental caches are particularly susceptible, so you may want to turn that off if you have turned them on.
If you determine the current version is good or bad, run git bisect good or git bisect bad to mark that, and then repeat building and marking until finished.
Similar to cargo-bisect-rustc, git bisect supports scripting and lots of other goodies.
+Check out its documentation for more.
The cargo-bisect-rustc tool makes it super easy to find exactly when behavior has regressed in rustc.
+It automatically downloads rustc artifacts and tests them against a project you provide until it finds the regression.
The Installation chapter shows how to install cargo-bisect-rustc.
+For a quick introduction, see the Tutorial.
+Otherwise, start at the Basic usage chapter to learn how cargo-bisect-rustc works.
The basic method for installing cargo-bisect-rustc is:
cargo install cargo-bisect-rustc
+Additional options are described below.
+Besides having a working Rust installation, you may need a few other things installed on your system:
+libssl-dev on Ubuntu, openssl-devel on Fedora or Alpine)openssl package)If you’re having trouble using the system OpenSSL installation, it can be built from scratch. +The following will enable the vendored OpenSSL build:
+cargo install cargo-bisect-rustc --features git2/vendored-openssl
+Beware that this also requires perl and make to be installed.
RUST_SRC_REPOcargo-bisect-rustc needs to access the git log of the rust repo.
+You can set the default location of that when installing it:
RUST_SRC_REPO=/path/to/rust cargo install cargo-bisect-rustc
+See Rust source repo for more about configuring how cargo-bisect-rustc retrieves this information.
The cargo-bisect-rustc tool makes it super easy to find exactly when behavior has regressed in rustc.
+It automatically downloads rustc artifacts and tests them against a project you provide until it finds the regression.
The Installation chapter shows how to install cargo-bisect-rustc.
+For a quick introduction, see the Tutorial.
+Otherwise, start at the Basic usage chapter to learn how cargo-bisect-rustc works.
The cargo-bisect-rustc tool makes it super easy to find exactly when behavior has regressed in rustc.
+It automatically downloads rustc artifacts and tests them against a project you provide until it finds the regression.
The Installation chapter shows how to install cargo-bisect-rustc.
+For a quick introduction, see the Tutorial.
+Otherwise, start at the Basic usage chapter to learn how cargo-bisect-rustc works.
The basic method for installing cargo-bisect-rustc is:
cargo install cargo-bisect-rustc
+Additional options are described below.
+Besides having a working Rust installation, you may need a few other things installed on your system:
+libssl-dev on Ubuntu, openssl-devel on Fedora or Alpine)openssl package)If you’re having trouble using the system OpenSSL installation, it can be built from scratch. +The following will enable the vendored OpenSSL build:
+cargo install cargo-bisect-rustc --features git2/vendored-openssl
+Beware that this also requires perl and make to be installed.
RUST_SRC_REPOcargo-bisect-rustc needs to access the git log of the rust repo.
+You can set the default location of that when installing it:
RUST_SRC_REPO=/path/to/rust cargo install cargo-bisect-rustc
+See Rust source repo for more about configuring how cargo-bisect-rustc retrieves this information.
Using cargo-bisect-rustc simply involves running it inside a Cargo project that reproduces the regression:
cargo bisect-rustc
+++For a quick introduction, see the Tutorial.
+
cargo-bisect-rustc works by building a Cargo project, and detecting if it succeeds or fails.
+It will download and use nightly Rust toolchains.
+It begins with two nightly boundaries, known as the start where the project successfully builds (the baseline), and the end where it is known to fail (the regression).
+It will then do a binary search between those dates to find the nightly where the project started to fail.
Once it finds the nightly where it started to fail, cargo-bisect-rustc will then try to find the individual PR where it regressed.
+The Rust project keeps the builds of every merged PR for the last 167 days.
+If the nightly is within that range, then it will bisect between those PRs.
And even further, if the regression is in a rollup PR, then it will bisect the individual PRs within the rollup.
+This final bisection is only available for x86_64-unknown-linux-gnu since it is using the builds made for the rustc performance tracker.
cargo-bisect-rustc needs to read the git log of the rust-lang/rust repo in order to scan individual commits.
+See the Rust src repo chapter for details on how to configure how it finds the git repo.
Without setting any options, cargo-bisect-rustc will try to automatically find the start where the build succeeds and the end where it fails.
+This can take some time, depending on how far back it needs to scan.
+It is recommended to use the --start and --end CLI options to tell it where the boundaries are.
cargo bisect-rustc --start=2022-11-01 --end=2023-02-14
+See the Bisection boundaries chapter for more details on setting these options.
+By default, cargo-bisect-rustc assumes the start boundary successfully builds, and the end boundary fails to build.
+You can change this using the --regress CLI option.
+For example, you can tell it that the start should fail, and the end should pass.
+There are several options you can use with the --regress flag:
| Option | Start | End | Description |
|---|---|---|---|
error | Succeed | Fail | The default setting checks for a failure as the regression. |
success | Fail | Succeed | Reverses the check to find when something is fixed. |
ice | No ICE | ICE | Scans when an Internal Compiler Error (ICE) was introduced. |
non-ice | ICE | No ICE | Scans when an ICE was fixed. |
non-error | Non-ICE Failure | Succeed or ICE | Scans when an ill-formed program stops being properly rejected, or the compiler starts generating an ICE. |
See Scripting for customizing this behavior.
+By default, cargo-bisect-rustc runs cargo build.
+You can change which cargo command is run by passing additional arguments after --:
cargo bisect rustc -- test --test mytest
+You can use an arbitrary script for determining what is a baseline and regression.
+This is an extremely flexible option that allows you to perform any action automatically.
+Just pass the path to the script to the --script CLI command:
cargo bisect-rustc --script ./test.sh
+The script should exit 0 for the baseline, and nonzero for a regression.
+Since cargo-bisect-rustc sets RUSTUP_TOOLCHAIN (see Rustup toolchains), all you need to do is call cargo or rustc, and the script should automatically use the toolchain that is currently being tested.
#!/bin/sh
+
+set -ex
+
+# This checks that a warning is only printed once.
+# See https://github.com/rust-lang/rust/issues/88256 for a regression where it
+# started printing twice.
+
+OUTPUT=`cargo check 2>&1`
+COUNT=`echo "$OUTPUT" | grep -c "unnecessary parentheses"`
+test $COUNT -eq 1
+If you need to use the targets directly without using cargo in the script, they are available in $CARGO_TARGET_DIR/[release|debug]/..., since cargo-bisect-rustc sets $CARGO_TARGET_DIR.
Check out the examples chapters for several examples of how to use this option.
+cargo-bisect-rustc works by building a Cargo project and checking if it succeeds or fails.
+This tutorial walks through an example of this process.
Create a cargo project that demonstrates the regression. +Let’s use issue #53157 as an example:
+cargo new foo
+cd foo
+Edit src/main.rs with the example from the issue:
+macro_rules! m { + () => {{ + fn f(_: impl Sized) {} + f + }} +} + +fn main() { + fn f() -> impl Sized {}; + m!()(f()); +}
Since we are testing an old regression, also edit Cargo.toml to remove the edition = "2021" field which isn’t supported in these versions.
Then run cargo bisect-rustc --end=2018-08-04.
We need to provide the end point for this particular example because that’s an old regression already fixed on the latest nightlies. +We could also provide a start point if we know one; +that’s going to make it faster by avoiding scanning for the start. +For instance:
+cargo bisect-rustc --start=2018-05-07 --end=2018-08-04
+It will run cargo build in the project and check whether or not it fails.
+It will do a binary search between the start and end range to find exactly where the regression occurred.
++Note: You can also use the flag
+--regressto specify other common regression criteria, e.g.--regress=icefor internal compiler errors.
In our example, in just a few steps, we can we find that it stopped working on nightly-2018-07-30.
If the regression is recent enough, then it will print out a list of PRs that were committed on that date. +In this particular example, it is too old, so we’ll need to manually inspect the git log to see which PR’s were merged.
+If the nightly was within the last 167 days, then cargo-bisect-rustc will then start bisecting those individual PRs.
After finding potential candidates, you can go inspect those PRs to see which one is the likely cause. +In this case, since the ICE was in MIR const propagation, and #51361 is the likely candidate since it modified const evaluation.
+Pass/fail of cargo build may not be what you’re after.
+Perhaps the issue is an error message changed, so both the “good” and “bad” version will fail to
+compile, just with a different message.
+Or maybe something used to fail, and now erroneously passes.
+You can use the interactive feature with the --prompt flag to visually inspect a build and tell cargo-bisect-rustc what’s “good” and what’s “bad”.
+Let’s use issue #55036 as an example where an error message changed:
In Cargo.toml, remove the edition field (this example was before editions).
src/main.rs:
+struct Foo { + bar: i32 +} + +trait Baz { + fn f(Foo { bar }: Foo) {} +} + +fn main() {}
This historically emitted a bad error, was updated to emit a nice error (E0642 added in #53051), but then that nice error was lost somewhere (on the 2015 edition). +Let’s find where it was lost! +Grab the ranges between where it was added and where we know it fails:
+cargo bisect-rustc --prompt \
+ --start=2018-08-14 \
+ --end=2018-10-11
+At each step, cargo-bisect-rustc will show the output and ask you:
nightly-2018-08-14 finished with exit code Some(101).
+please select an action to take:
+> mark regressed
+ mark baseline
+ retry
+Choose mark baseline with the nice E0642 message, and mark regressed with the less-favorable token error.
+Fairly quickly we find it regressed in nightly-2018-10-11.
+The most likely candidate is #54457 which is a rollup PR.
+It’s usually not too hard to look through the commits and find a likely culprit.
+Indeed in this example, #54415 modified function parameter parsing.
Using the --script option allows you to do something more fancy than just cargo build.
+Maybe you need to run cargo multiple times, or just call rustc directly, or you want to automatically grep through the output.
+The possibilities are endless!
+Just write a little shell script that exits 0 for the baseline, and exits nonzero for the regression.
+As an example, the previous interactive session can be hands-free automated with this script:
test.sh:
#!/bin/sh
+
+# Fail if we no longer get a `E0642` error:
+cargo check 2>&1 | grep E0642
+And then run:
+cargo bisect-rustc --script=./test.sh \
+ --start=2018-08-14 \
+ --end=2018-10-11
+Available from v0.6.9
+You can add custom messages when bisecting a regression. Taking inspiration from git-bisect, with term-new and term-old you can set custom messages to indicate if a regression matches the condition set by the bisection.
Example:
+cargo bisect-rustc \
+ --start=2018-08-14 \
+ --end=2018-10-11 \
+ --term-old "No, this build did not reproduce the regression, compile successful" \
+ --term-new "Yes, this build reproduces the regression, compile error"
+In other words, --term-old is displayed for older compilers that do not exhibit the regression. --term-new is for newer compilers which do exhibit the regression.
What counts as a “regression” is defined by the --regress CLI option. By default, a regression is a compile-error (which is equivalent to --term-new). If you flip the definition of a “regression” with --regress=success, then a regression is a successful compile (which is also equivalent to --term-new).
There are default terms based on the current --regress setting. Customizing the terms is most useful when using scripting. For example, in the Documentation changes example, the customized terms can more clearly express the results of the script of whether or not it found what it was looking for in the documentation.
For cargo-bisect-rustc to work, it needs to be able to read the git log of the rust-lang/rust repo.
+cargo-bisect-rustc supports several methods for this described below.
By default, cargo-bisect-rustc uses the GitHub API to fetch the information instead of using a local checkout.
cargo bisect-rustc --access=github
+Beware that GitHub has restrictive rate limits for unauthenticated requests.
+It allows 60 requests per hour, and cargo-bisect-rustc will use about 10 requests each time you run it (which can vary depending on the bisection).
+If you run into the rate limit, you can raise it to 5000 requests per hour by setting the GITHUB_TOKEN environment variable to a GitHub personal token.
+If you use the gh CLI tool, you can use it to get a token:
GITHUB_TOKEN=`gh auth token` cargo bisect-rustc --access=github
+If you don’t use gh, you’ll just need to copy and paste the token.
cargo-bisect-rustc can also clone the rust repo in the current directory (as rust.git).
+This option can be quite slow if you don’t specify the repo path at build time.
+You can specify this with the --access CLI argument:
cargo bisect-rustc --access=checkout
+RUST_SRC_REPO environment variableYou can specify the location of the rust repo with the RUST_SRC_REPO environment variable at runtime.
+This is useful if you already have it checked out somewhere, but is cumbersome to use.
RUST_SRC_REPO=/path/to/rust cargo bisect-rustc
+RUST_SRC_REPO environment variable (build-time)Setting the RUST_SRC_REPO environment variable when installing cargo-bisect-rustc will set the default location for the rust repo.
+This is recommended if you already have the rust repo checked out somewhere.
RUST_SRC_REPO=/path/to/rust cargo install cargo-bisect-rustc
+cargo-bisect-rustc does a binary search for the regression using a start and end boundary.
+You can specify these boundaries with the --start and --end CLI flags.
+There are several ways to specify what those boundaries are.
+If you run the command without specifying the boundaries, it will search for them automatically:
# No --start or --end flags
+cargo bisect-rustc
+This will assume the latest nightly is a regression (the end boundary). +It will then search backwards until it can find a nightly that passes to use as the start boundary. +Bisection can usually go faster if you happen to know the start boundary, so that it doesn’t need to search for it.
+--start and --end are optional.
+If --start is not specified, then it will try to find the start range automatically.
+If --end is not specified, it will assume it is the most recently available.
You can pass a date in the form YYYY-MM-DD to the --start and --end flags.
+It will download the nightly corresponding to that date, and then begin bisecting those nightlies.
cargo bisect-rustc --start=2018-08-14 --end=2018-10-11
+If the nightly with the regression was within the past 167 days, then it will automatically start bisecting the individual PRs merged on that day using Git commit boundaries.
+You can pass the particular git commit hash of a PR as a boundary. +The Rust project keeps the builds of every merged PR for the last 167 days. +If you happen to know the PR to use as a boundary, you can pass the SHA-1 hash of that PR.
+cargo bisect-rustc \
+ --start=6323d9a45bdf0ac2a9319a6a558537e0a7e6abd1 \
+ --end=866a713258915e6cbb212d135f751a6a8c9e1c0a
+There are several ways to determine the SHA-1 hash for a PR.
+rust-lang:master”.
+You can copy that hash to use as a boundary.
+If the PR was merged as part of a rollup, you will need to use the hash of the rollup instead.
+You’ll need to look through the PR messages to see if the PR was mentioned from a rollup PR.git log --first-parent upstream/master (where upstream is your origin name for rust-lang/rust).
+This will show all the top-level commits.
+You can then search for your PR.++Note: If the PR was merged after the most recent nightly, you’ll need to be sure to also specify the
+--endrange. +Otherwise it will assume the most recent nightly is the end and it won’t work if the start is after the end.
If the regression is found in a rollup PR, then cargo-bisect-rustc will bisect the individual PRs within the rollup.
+This final bisection is only available for x86_64-unknown-linux-gnu since it is using the builds made for the rustc performance tracker.
++Note: If you specify date boundaries, then you can use the
+--by-commitCLI option to force it to use PR commits instead of nightlies.
The boundary can be specified with a git release tag.
+This is useful if you know something works in one release and not another, but you don’t happen to know which nightly this corresponds with.
+When given a tag, cargo-bisect-rustc will try to find the nightly that corresponds with that release.
+For example:
cargo bisect-rustc --start=1.58.0 --end=1.59.0
+When writing your test and picking a bisection range, you should be careful to ensure that the test won’t vary between pass/fail over the bisection range. +It should only transition from good to bad once in the bisection range (it must change +monotonically).
+In the following example, cargo-bisect-rustc will find one of the transitions, but that may not be the true root cause of the issue you are investigating.
nightly-2023-02-01 baseline **start**
+nightly-2023-02-02 baseline
+nightly-2023-02-03 baseline
+nightly-2023-02-04 regression
+nightly-2023-02-05 regression
+nightly-2023-02-06 baseline
+nightly-2023-02-07 regression
+nightly-2023-02-08 regression **end**
+Here it may either find 2023-02-04 or 2023-02-07 as the regression.
+The following are some suggestions for avoiding or dealing with this problem:
+-vv flag (very verbose) to display the output from the compiler to make sure it is what you expect.--prompt flag to inspect the output and verify each step.cargo-bisect-rustc takes advantage of rustup toolchains for installation and selecting the correct rustc to run.
+It will essentially run cargo +bisector-nightly-2023-03-18-x86_64-unknown-linux-gnu build using rustup toolchain override shorthand to run the toolchains that it downloads.
+This sets the RUSTUP_TOOLCHAIN environment variable to the toolchain name, which ensures that any call to rustc will use the correct toolchain.
By default, cargo-bisect-rustc will delete toolchains immediately after using them.
+You can use the --preserve option to keep the toolchains so that you can use them manually.
+See the Preserving toolchains example for more details.
When using the --script option, the script should just invoke cargo or rustc normally, and rely on the RUSTUP_TOOLCHAIN environment variable to pick the correct toolchain.
There are some rare cases where you may need to build rustc with custom options, or otherwise work around issues with pre-built compilers not being available.
+For this you can use git bisect to build the compiler locally.
It can be helpful to use the --first-parent option so that it only bisects the merge commits directly reachable on the master branch.
+Otherwise the bisecting may land on intermediate commits from within a PR which may not build or test correctly.
To start the bisection, specifying the boundaries where the bisection will start:
+git bisect start --first-parent
+git bisect good 96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef
+git bisect bad a00f8ba7fcac1b27341679c51bf5a3271fa82df3
+Then, build the compiler as needed and run your tests to check for a regression:
+./x.py build std
+rustc +stage1 foo.rs
+You may want to consider running ./x.py clean if you are running into issues since changes to the internal structures of build artifacts aren’t always versioned, and those changes can be incompatible.
+Incremental caches are particularly susceptible, so you may want to turn that off if you have turned them on.
If you determine the current version is good or bad, run git bisect good or git bisect bad to mark that, and then repeat building and marking until finished.
Similar to cargo-bisect-rustc, git bisect supports scripting and lots of other goodies.
+Check out its documentation for more.
Each commit also generates what are called “alt” builds. +These are builds of rustc with some different options set. +As of August 2023, these include:
+rust.parallel-compilerllvm.assertionsrust.verify-llvm-irFor more information on these settings, see the config.toml docs.
+These alt settings are defined in ci/run.sh.
Alt builds are only available for a few targets.
+Look for the -alt builds in ci.yml.
This can be useful if you are bisecting an LLVM issue. +With LLVM assertions enabled, alt builds have checks that can help identify broken assumptions.
+Alt builds are only made for commit builds, and not nightly releases.
+You will need to specify --by-commit (or use a hash in the --start or --end flags) to only use commit builds.
cargo bisect-rustc --alt --by-commit
+The following chapters show examples of different ways of using cargo-bisect-rustc.
The following is an example of checking when the diagnostic output of rustc changes.
+For example, this can check when either the wording has changed, or a different error or warning is produced.
#109067 is an example of where this is necessary.
+A warning started being emitted, and it is the kind of warning that cannot be turned into an error with deny(warnings).
The following script is intended to be used with the --script option (set the executable flag on the script, chmod u+x):
#!/bin/sh
+
+OUTPUT=`cargo check 2>&1`
+# Comment out this test if your example is intended to fail.
+if [ $? -ne 0 ]
+then
+ echo "Build unexpectedly failed: $OUTPUT"
+ exit 1
+fi
+# Display the output for debugging purposes.
+# Run `cargo-bisect-rustc` with `-vv` to view the output.
+echo "$OUTPUT"
+# This indicates a regression when the text "non-ASCII" is in the output.
+#
+# If the regression is when the text is *not* in the output, remove the `!` prefix
+# (and customize the `--term-old` and `--term-new` CLI options if you want).
+! echo "$OUTPUT" | grep "non-ASCII"
+Then run something like:
+cargo bisect-rustc --start=1.67.0 --end=1.68.0 --script ./test.sh \
+ --term-old="No warning" --term-new="Found non-ASCII warning"
+Using the --script option on Windows can be cumbersome because Windows does not support #! scripts like Unix does, and the built-in scripting can also be awkward.
+The following sections show the different ways you can use scripting.
You can use DOS-style .bat files:
test.bat:
(cargo check 2>&1) | find "E0642"
+This can be executed directly with:
+cargo-bisect-rustc --script ./test.bat
+But .bat can be challenging to do more complex options, or you may not be familiar with it.
You can’t execute .ps1 Powershell files directly, so you will need to use pwsh to launch them:
test.ps1:
( cargo check 2>&1 ) | grep E0642
+if ( -Not $? ) {
+ exit 1
+}
+This can be run with:
+cargo-bisect-rustc --script pwsh -- -File ./test.ps1
+If you have Git-for-Windows installed, then you can use its copy of bash to run bash scripts:
+test.sh:
#!/bin/bash
+
+cargo check 2>&1 | grep E0642
+This can be run with:
+cargo-bisect-rustc --script "C:\\Program Files\\Git\\usr\\bin\\bash.exe" -- ./test.sh
+This also works if you have bash from something like msys2 installed.
+Testing for regressions with incremental compilation may require running a command multiple times. +The following illustrates an example for #87384 which only generates a warning the second time a build is run with incremental. +Previously no warning was emitted.
+foo.rs:
+#![type_length_limit = "95595489"] + +pub fn main() { + println!("Hello, world!"); +}
Create a script test.sh:
#!/bin/sh
+
+# Exit if any command fails.
+set -e
+
+rm -rf incremental
+rustc foo.rs --crate-type lib -C incremental=incremental
+echo second
+OUTPUT=`rustc foo.rs --crate-type lib -C incremental=incremental 2>&1`
+echo $OUTPUT
+! echo "$OUTPUT" | grep \
+ "crate-level attribute should be in the root module"
+Run this script with:
+cargo-bisect-rustc --start 1.54.0 --end 1.55.0 --script ./test.sh
+Some regressions may involve the compiler hanging or taking an unusually long time to run.
+The --timeout CLI option can be used to check for this.
+Let’s use #89524 as an example.
+A particular combination of factors caused the compiler to start to hang.
Change Cargo.toml to the following:
[package]
+name = "slow"
+version = "0.1.0"
+
+[dependencies]
+config = "=0.9.3"
+
+[profile.release]
+panic = "abort"
+codegen-units = 1
+Then use the timeout option:
+cargo-bisect-rustc --start=2021-09-01 --end=2021-10-02 --timeout 30 -- build --release
+You may need to adjust the timeout value based on the speed of your system.
+++Note:
+--timeoutis currently not working on macOS. See https://github.com/rust-lang/cargo-bisect-rustc/issues/232.
By default, cargo-bisect-rustc only fetches rustc, cargo, rustdoc, and the standard library for the host.
+You may need additional Rustup Components to run your test.
+Some examples of when this might be needed are:
x86_64-pc-windows-gnu host may need the rust-mingw component.If you are testing cross-compilation, use the --target option to download the standard library for the target you are using.
The following example shows how to use components to do a bisection with Cargo’s build-std feature.
+cargo-bisect-rustc --start=2022-11-01 --end=2022-11-20 -c rust-src -- build -Zbuild-std
+++Note: The
+--with-srcoption is an alias for-c rust-src.
+The--with-devoption is an alias for-c rustc-dev -c llvm-tools.
Some bisections don’t require Cargo.
+You can use the --without-cargo option to skip installing cargo which can speed up the bisection since it doesn’t need to download cargo, and doesn’t have the overhead of running cargo.
+You will need to pair this with --script since cargo-bisect-rustc assumes projects use Cargo.
For example, using a simple rustc command:
cargo-bisect-rustc --start=2022-11-01 --end=2022-11-20 --without-cargo --script=rustc -- foo.rs
+++Note: You can use
+--without-cargowhile still using a Cargo project. +Rustup will fall back to usingcargofrom your installed nightly, beta, or stable toolchain. +However, this isn’t recommended sincecargois only intended to work with the version it is released with, and can sometimes be incompatible with different versions. +But if you are bisecting a very recent change, then you can probably get away with it.
You may want to reuse the toolchains downloaded by cargo-bisect-rustc for doing further analysis or debugging.
+Or, while setting up your regression test, you may need to adjust your test and script several times, and downloading the same toolchains multiple times can be quite slow.
You can do this with the --preserve option.
cargo bisect-rustc --start=2023-01-01 --end=2023-02-01 --preserve
+The toolchains will be kept in your Rustup home directory (typically ~/.rustup/toolchains).
Toolchains for nightlies will have the form of bisector-nightly-YYYY-MM-DD-<target>.
+Toolchains for PR artifacts will have the form of bisector-ci-<hash>-<target>.
You can run these toolchains using a Rustup override, like this:
+cargo +bisector-nightly-2023-03-18-x86_64-unknown-linux-gnu build
+# or...
+cargo +bisector-ci-e187f8871e3d553181c9d2d4ac111197a139ca0d-x86_64-unknown-linux-gnu build
+When you are done, you’ll probably want to clean up these directories since they use a lot of space. +The easiest method is to just delete the directories:
+rm -rf ~/.rustup/toolchains/bisector-*
+The --install option can be used to only install a toolchain.
+This won’t do a bisection, it is just for fetching a toolchain for testing.
cargo bisect-rustc --install e187f8871e3d553181c9d2d4ac111197a139ca0d
+++Note: See also
+rustup-toolchain-install-masterwhich is specialized for installing CI artifacts.
cargo-bisect-rustc can be used to check for Rustdoc regressions, too.
+All you need to do is instruct it to use the correct command.
The following example will check to find a regression when cargo doc suddenly starts to fail.
cargo bisect-rustc --start=2022-08-05 --end=2022-09-09 -- doc
+Some rustdoc regressions might be in the generated HTML output. +To scan the output, you can use a script like the following:
+test.sh:
#!/bin/sh
+
+# Exit if any command fails.
+set -e
+
+cargo doc
+
+grep "some example text" $CARGO_TARGET_DIR/doc/mycrate/fn.foo.html
+This can be used with the --script option:
cargo-bisect-rustc --start=2023-01-22 --end=2023-03-18 --script=./test.sh \
+ --term-old="Found example text" --term-new="Failed, or did not find text"
+cargo-bisect-rustc can be used to check for Clippy regressions, too.
+You’ll need to instruct it to download clippy, and run the command correctly:
cargo bisect-rustc --start=1.67.0 --end=1.68.0 -c clippy -- clippy
+Note that depending on what you are looking for, this may just find a PR that syncs the rust-clippy repo to rust-lang/rust.
+You may be able to scan the list of changes in that PR to discover what you are looking for.
+If the list of changes is too big or nothing is jumping out as a possible culprit, then consider using git bisect on the clippy repo itself (which will require building clippy).
cargo-bisect-rustc can be used to scan for changes in the documentation shipped with each release.
+This includes all the books and standard library documentation.
+To do this, instruct it to download the component, and use a script that scans for whatever you are looking for.
+You can use rustup doc --path or rustc --print=sysroot to find the proper location.
+For example:
test.sh:
#!/bin/sh
+
+# Exit if any command fails.
+set -e
+
+STD=`dirname $(rustup doc --std --path)`
+
+# Checks if a particular file exists.
+# This could also be `grep` or any other kinds of tests you need.
+if [ -e $STD/io/error/type.RawOsError.html ]
+then
+ echo "found"
+ exit 1
+fi
+And run with:
+cargo bisect-rustc --start 1.68.0 --end 1.69.0 -c rust-docs --script ./test.sh \
+ --term-old="Did not find" --term-new="Found"
+++Note: This may not work on all targets since
+cargo-bisect-rustcdoesn’t properly handle rustup manifests, which alias some targets to other targets. +Use--host x86_64-unknown-linux-gnuin that situation.
Some tests may fail randomly.
+The following script is an example that will run rustc repeatedly to check for a failure.
+This example is from #108216 (which requires macOS).
test.sh:
#!/bin/sh
+
+rm -rf *.o incremental foo
+
+echo "fn main() { let a: i64 = 1 << 64; }" > foo1.rs
+echo "fn main() { let a: i64 = 1 << 63; }" > foo2.rs
+
+ARGS="--crate-name foo -C split-debuginfo=unpacked -C debuginfo=2 -C incremental=incremental"
+
+for i in {1..20}
+do
+ echo run $i
+ rustc foo1.rs $ARGS && { echo "ERROR: first build should have failed"; exit 1; }
+ rustc foo2.rs $ARGS || { echo "ERROR: second build should have passed"; exit 1; }
+ ./foo || { echo "ERROR: executing should have passed"; exit 1; }
+done
+This test can be run with:
+cargo bisect-rustc --start=1.57.0 --end=1.58.0 --script=./test.sh
+In general, configure the script to perform whichever actions you need in a for loop that runs enough times that you have a high confidence it has found the regression.
For cargo-bisect-rustc to work, it needs to be able to read the git log of the rust-lang/rust repo.
+cargo-bisect-rustc supports several methods for this described below.
By default, cargo-bisect-rustc uses the GitHub API to fetch the information instead of using a local checkout.
cargo bisect-rustc --access=github
+Beware that GitHub has restrictive rate limits for unauthenticated requests.
+It allows 60 requests per hour, and cargo-bisect-rustc will use about 10 requests each time you run it (which can vary depending on the bisection).
+If you run into the rate limit, you can raise it to 5000 requests per hour by setting the GITHUB_TOKEN environment variable to a GitHub personal token.
+If you use the gh CLI tool, you can use it to get a token:
GITHUB_TOKEN=`gh auth token` cargo bisect-rustc --access=github
+If you don’t use gh, you’ll just need to copy and paste the token.
cargo-bisect-rustc can also clone the rust repo in the current directory (as rust.git).
+This option can be quite slow if you don’t specify the repo path at build time.
+You can specify this with the --access CLI argument:
cargo bisect-rustc --access=checkout
+RUST_SRC_REPO environment variableYou can specify the location of the rust repo with the RUST_SRC_REPO environment variable at runtime.
+This is useful if you already have it checked out somewhere, but is cumbersome to use.
RUST_SRC_REPO=/path/to/rust cargo bisect-rustc
+RUST_SRC_REPO environment variable (build-time)Setting the RUST_SRC_REPO environment variable when installing cargo-bisect-rustc will set the default location for the rust repo.
+This is recommended if you already have the rust repo checked out somewhere.
RUST_SRC_REPO=/path/to/rust cargo install cargo-bisect-rustc
+cargo-bisect-rustc takes advantage of rustup toolchains for installation and selecting the correct rustc to run.
+It will essentially run cargo +bisector-nightly-2023-03-18-x86_64-unknown-linux-gnu build using rustup toolchain override shorthand to run the toolchains that it downloads.
+This sets the RUSTUP_TOOLCHAIN environment variable to the toolchain name, which ensures that any call to rustc will use the correct toolchain.
By default, cargo-bisect-rustc will delete toolchains immediately after using them.
+You can use the --preserve option to keep the toolchains so that you can use them manually.
+See the Preserving toolchains example for more details.
When using the --script option, the script should just invoke cargo or rustc normally, and rely on the RUSTUP_TOOLCHAIN environment variable to pick the correct toolchain.
cargo-bisect-rustc works by building a Cargo project and checking if it succeeds or fails.
+This tutorial walks through an example of this process.
Create a cargo project that demonstrates the regression. +Let’s use issue #53157 as an example:
+cargo new foo
+cd foo
+Edit src/main.rs with the example from the issue:
+macro_rules! m { + () => {{ + fn f(_: impl Sized) {} + f + }} +} + +fn main() { + fn f() -> impl Sized {}; + m!()(f()); +}
Since we are testing an old regression, also edit Cargo.toml to remove the edition = "2021" field which isn’t supported in these versions.
Then run cargo bisect-rustc --end=2018-08-04.
We need to provide the end point for this particular example because that’s an old regression already fixed on the latest nightlies. +We could also provide a start point if we know one; +that’s going to make it faster by avoiding scanning for the start. +For instance:
+cargo bisect-rustc --start=2018-05-07 --end=2018-08-04
+It will run cargo build in the project and check whether or not it fails.
+It will do a binary search between the start and end range to find exactly where the regression occurred.
++Note: You can also use the flag
+--regressto specify other common regression criteria, e.g.--regress=icefor internal compiler errors.
In our example, in just a few steps, we can we find that it stopped working on nightly-2018-07-30.
If the regression is recent enough, then it will print out a list of PRs that were committed on that date. +In this particular example, it is too old, so we’ll need to manually inspect the git log to see which PR’s were merged.
+If the nightly was within the last 167 days, then cargo-bisect-rustc will then start bisecting those individual PRs.
After finding potential candidates, you can go inspect those PRs to see which one is the likely cause. +In this case, since the ICE was in MIR const propagation, and #51361 is the likely candidate since it modified const evaluation.
+Pass/fail of cargo build may not be what you’re after.
+Perhaps the issue is an error message changed, so both the “good” and “bad” version will fail to
+compile, just with a different message.
+Or maybe something used to fail, and now erroneously passes.
+You can use the interactive feature with the --prompt flag to visually inspect a build and tell cargo-bisect-rustc what’s “good” and what’s “bad”.
+Let’s use issue #55036 as an example where an error message changed:
In Cargo.toml, remove the edition field (this example was before editions).
src/main.rs:
+struct Foo { + bar: i32 +} + +trait Baz { + fn f(Foo { bar }: Foo) {} +} + +fn main() {}
This historically emitted a bad error, was updated to emit a nice error (E0642 added in #53051), but then that nice error was lost somewhere (on the 2015 edition). +Let’s find where it was lost! +Grab the ranges between where it was added and where we know it fails:
+cargo bisect-rustc --prompt \
+ --start=2018-08-14 \
+ --end=2018-10-11
+At each step, cargo-bisect-rustc will show the output and ask you:
nightly-2018-08-14 finished with exit code Some(101).
+please select an action to take:
+> mark regressed
+ mark baseline
+ retry
+Choose mark baseline with the nice E0642 message, and mark regressed with the less-favorable token error.
+Fairly quickly we find it regressed in nightly-2018-10-11.
+The most likely candidate is #54457 which is a rollup PR.
+It’s usually not too hard to look through the commits and find a likely culprit.
+Indeed in this example, #54415 modified function parameter parsing.
Using the --script option allows you to do something more fancy than just cargo build.
+Maybe you need to run cargo multiple times, or just call rustc directly, or you want to automatically grep through the output.
+The possibilities are endless!
+Just write a little shell script that exits 0 for the baseline, and exits nonzero for the regression.
+As an example, the previous interactive session can be hands-free automated with this script:
test.sh:
#!/bin/sh
+
+# Fail if we no longer get a `E0642` error:
+cargo check 2>&1 | grep E0642
+And then run:
+cargo bisect-rustc --script=./test.sh \
+ --start=2018-08-14 \
+ --end=2018-10-11
+Available from v0.6.9
+You can add custom messages when bisecting a regression. Taking inspiration from git-bisect, with term-new and term-old you can set custom messages to indicate if a regression matches the condition set by the bisection.
Example:
+cargo bisect-rustc \
+ --start=2018-08-14 \
+ --end=2018-10-11 \
+ --term-old "No, this build did not reproduce the regression, compile successful" \
+ --term-new "Yes, this build reproduces the regression, compile error"
+In other words, --term-old is displayed for older compilers that do not exhibit the regression. --term-new is for newer compilers which do exhibit the regression.
What counts as a “regression” is defined by the --regress CLI option. By default, a regression is a compile-error (which is equivalent to --term-new). If you flip the definition of a “regression” with --regress=success, then a regression is a successful compile (which is also equivalent to --term-new).
There are default terms based on the current --regress setting. Customizing the terms is most useful when using scripting. For example, in the Documentation changes example, the customized terms can more clearly express the results of the script of whether or not it found what it was looking for in the documentation.
Using cargo-bisect-rustc simply involves running it inside a Cargo project that reproduces the regression:
cargo bisect-rustc
+++For a quick introduction, see the Tutorial.
+
cargo-bisect-rustc works by building a Cargo project, and detecting if it succeeds or fails.
+It will download and use nightly Rust toolchains.
+It begins with two nightly boundaries, known as the start where the project successfully builds (the baseline), and the end where it is known to fail (the regression).
+It will then do a binary search between those dates to find the nightly where the project started to fail.
Once it finds the nightly where it started to fail, cargo-bisect-rustc will then try to find the individual PR where it regressed.
+The Rust project keeps the builds of every merged PR for the last 167 days.
+If the nightly is within that range, then it will bisect between those PRs.
And even further, if the regression is in a rollup PR, then it will bisect the individual PRs within the rollup.
+This final bisection is only available for x86_64-unknown-linux-gnu since it is using the builds made for the rustc performance tracker.
cargo-bisect-rustc needs to read the git log of the rust-lang/rust repo in order to scan individual commits.
+See the Rust src repo chapter for details on how to configure how it finds the git repo.
Without setting any options, cargo-bisect-rustc will try to automatically find the start where the build succeeds and the end where it fails.
+This can take some time, depending on how far back it needs to scan.
+It is recommended to use the --start and --end CLI options to tell it where the boundaries are.
cargo bisect-rustc --start=2022-11-01 --end=2023-02-14
+See the Bisection boundaries chapter for more details on setting these options.
+By default, cargo-bisect-rustc assumes the start boundary successfully builds, and the end boundary fails to build.
+You can change this using the --regress CLI option.
+For example, you can tell it that the start should fail, and the end should pass.
+There are several options you can use with the --regress flag:
| Option | Start | End | Description |
|---|---|---|---|
error | Succeed | Fail | The default setting checks for a failure as the regression. |
success | Fail | Succeed | Reverses the check to find when something is fixed. |
ice | No ICE | ICE | Scans when an Internal Compiler Error (ICE) was introduced. |
non-ice | ICE | No ICE | Scans when an ICE was fixed. |
non-error | Non-ICE Failure | Succeed or ICE | Scans when an ill-formed program stops being properly rejected, or the compiler starts generating an ICE. |
See Scripting for customizing this behavior.
+By default, cargo-bisect-rustc runs cargo build.
+You can change which cargo command is run by passing additional arguments after --:
cargo bisect rustc -- test --test mytest
+You can use an arbitrary script for determining what is a baseline and regression.
+This is an extremely flexible option that allows you to perform any action automatically.
+Just pass the path to the script to the --script CLI command:
cargo bisect-rustc --script ./test.sh
+The script should exit 0 for the baseline, and nonzero for a regression.
+Since cargo-bisect-rustc sets RUSTUP_TOOLCHAIN (see Rustup toolchains), all you need to do is call cargo or rustc, and the script should automatically use the toolchain that is currently being tested.
#!/bin/sh
+
+set -ex
+
+# This checks that a warning is only printed once.
+# See https://github.com/rust-lang/rust/issues/88256 for a regression where it
+# started printing twice.
+
+OUTPUT=`cargo check 2>&1`
+COUNT=`echo "$OUTPUT" | grep -c "unnecessary parentheses"`
+test $COUNT -eq 1
+If you need to use the targets directly without using cargo in the script, they are available in $CARGO_TARGET_DIR/[release|debug]/..., since cargo-bisect-rustc sets $CARGO_TARGET_DIR.
Check out the examples chapters for several examples of how to use this option.
+ +