Use actual PR summaries instead of bors placeholders when listing them #379

moxian · 2025-05-03T21:28:54Z

When bisecting issues it often happens that a regression was introduced long enough ago that CI artifacts are unavailable, and we only have a list of PRs between two nightlies to go off of.

The current output only presents a numeric(!) list of PRs, which makes it very cumbersome to inspect those and narrow down. Best way I found was to handle this was to start a new comment on the issue, paste cargo-bisect-rustc output, then go to preview tab, and open all of the hotlinked issue numbers in new tabs.

This can be improved.
We can parse the bors autogenerated commit description and extract the original PR summary, and use that instead of just a number.
We can go even further and explicitly list all of the individual rolled-up PRs in case of rollups.

Output on current master (for mcve in rust-lang/rust#139089 as a random example):

> cargo bisect-rustc --start=2024-02-08 --end=2024-02-09 --regress=ice --preserve
checking the start range to find a passing nightly
installing nightly-2024-02-08
testing...
RESULT: nightly-2024-02-08, ===> Did not ICE

checking the end range to verify it does not pass
installing nightly-2024-02-09
testing...
RESULT: nightly-2024-02-09, ===> Found ICE

searched toolchains nightly-2024-02-08 through nightly-2024-02-09
checking last toolchain to determine final result
installing nightly-2024-02-09
testing...


********************************************************************************
Regression in nightly-2024-02-09
********************************************************************************

fetching https://static.rust-lang.org/dist/2024-02-08/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2024-02-08: 40 B / 40 B [======================================================] 100.00 % 123.73 KB/s converted 2024-02-08 to 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6
fetching https://static.rust-lang.org/dist/2024-02-09/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2024-02-09: 40 B / 40 B [======================================================] 100.00 % 155.07 KB/s converted 2024-02-09 to 98aa3624be70462d6a25ed5544333e3df62f4c66
looking for regression commit between 2024-02-08 and 2024-02-09
fetching (via remote github) commits from max(8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6, 2024-02-06) to 98aa3624be70462d6a25ed5544333e3df62f4c66
ending github query because we found starting sha: 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6
get_commits_between returning commits, len: 9
  commit[0] 2024-02-07: Auto merge of #120748 - Nadrieril:rollup-dj0qwv5, r=Nadrieril
  commit[1] 2024-02-08: Auto merge of #120381 - fee1-dead-contrib:reconstify-add, r=compiler-errors
  commit[2] 2024-02-08: Auto merge of #120521 - reitermarkus:generic-nonzero-constructors, r=dtolnay
  commit[3] 2024-02-08: Auto merge of #120558 - oli-obk:missing_impl_item_ice, r=estebank
  commit[4] 2024-02-08: Auto merge of #120579 - GuillaumeGomez:prevent-running-unneeded-code, r=notriddle
  commit[5] 2024-02-08: Auto merge of #120550 - oli-obk:track_errors8, r=estebank
  commit[6] 2024-02-08: Auto merge of #120767 - matthiaskrgr:rollup-0k8ib1c, r=matthiaskrgr
  commit[7] 2024-02-08: Auto merge of #120544 - BoxyUwU:enter_forall, r=lcnr
  commit[8] 2024-02-08: Auto merge of #120807 - matthiaskrgr:rollup-1pf3glu, r=matthiaskrgr
ERROR: no CI builds available between 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6 and 98aa3624be70462d6a25ed5544333e3df62f4c66 within last 167 days

Output with the PR:

> cargo bisect-rustc --start=2024-02-08 --end=2024-02-09 --regress=ice --preserve
checking the start range to find a passing nightly
installing nightly-2024-02-08
testing...
RESULT: nightly-2024-02-08, ===> Did not ICE

checking the end range to verify it does not pass
installing nightly-2024-02-09
testing...
RESULT: nightly-2024-02-09, ===> Found ICE

searched toolchains nightly-2024-02-08 through nightly-2024-02-09
checking last toolchain to determine final result
installing nightly-2024-02-09
testing...


********************************************************************************
Regression in nightly-2024-02-09
********************************************************************************

fetching https://static.rust-lang.org/dist/2024-02-08/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2024-02-08: 40 B / 40 B [=======================================================] 100.00 % 76.20 KB/s converted 2024-02-08 to 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6
fetching https://static.rust-lang.org/dist/2024-02-09/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2024-02-09: 40 B / 40 B [=======================================================] 100.00 % 96.31 KB/s converted 2024-02-09 to 98aa3624be70462d6a25ed5544333e3df62f4c66
looking for regression commit between 2024-02-08 and 2024-02-09
fetching (via remote github) commits from max(8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6, 2024-02-06) to 98aa3624be70462d6a25ed5544333e3df62f4c66
ending github query because we found starting sha: 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6
PRs in range:
  - #120748 (Rollup of 13 pull requests) by Nadrieril
    - #110482 (Add armv8r-none-eabihf target for the Cortex-R52.)
    - #119162 (Add unstable `-Z direct-access-external-data` cmdline flag for `rustc`)
    - #120302 (various const interning cleanups)
    - #120455 ( Add FileCheck annotations to MIR-opt SROA tests)
    - #120470 (Mark "unused binding" suggestion as maybe incorrect)
    - #120479 (Suggest turning `if let` into irrefutable `let` if appropriate)
    - #120564 (coverage: Split out counter increment sites from BCB node/edge counters)
    - #120633 (pattern_analysis: gather up place-relevant info)
    - #120664 (Add parallel rustc ui tests)
    - #120726 (Don't use bashism in checktools.sh)
    - #120733 (MirPass: make name more const)
    - #120735 (Remove some `unchecked_claim_error_was_emitted` calls)
    - #120746 (Record coroutine kind in coroutine generics)
  - #120381 (Reconstify `Add`) by fee1-dead-contrib
  - #120521 (Make `NonZero` constructors generic.) by reitermarkus
  - #120558 (Stop bailing out from compilation just because there were incoherent traits) by oli-obk
  - #120579 (Prevent running some code if it is already in the map) by GuillaumeGomez
  - #120550 (Continue to borrowck even if there were previous errors) by oli-obk
  - #120767 (Rollup of 9 pull requests) by matthiaskrgr
    - #119592 (resolve: Unload speculatively resolved crates before freezing cstore)
    - #120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
    - #120206 (hir: Make sure all `HirId`s have corresponding HIR `Node`s)
    - #120214 (match lowering: consistently lower bindings deepest-first)
    - #120688 (GVN: also turn moves into copies with projections)
    - #120702 (docs: also check the inline stmt during redundant link check)
    - #120727 (exhaustiveness: Prefer "`0..MAX` not covered" to "`_` not covered")
    - #120734 (Add `SubdiagnosticMessageOp` as a trait alias.)
    - #120739 (improve pretty printing for associated items in trait objects)
  - #120544 (Introduce `enter_forall` to supercede `instantiate_binder_with_placeholders`) by BoxyUwU
  - #120807 (Rollup of 9 pull requests) by matthiaskrgr
    - #120590 (Remove unused args from functions)
    - #120750 (No need to take `ImplTraitContext` by ref)
    - #120769 (make future diffs minimal)
    - #120772 (Remove myself from review rotation.)
    - #120775 (Make `min_exhaustive_patterns` match `exhaustive_patterns` better)
    - #120778 (Deduplicate `tcx.instance_mir(instance)` calls in `try_instance_mir`)
    - #120782 (Fix mir pass ICE in the presence of other errors)
    - #120783 (Add release note for new ambiguous_wide_pointer_comparisons lint)
    - #120801 (Avoid ICE in drop recursion check in case of invalid drop impls)
ERROR: no CI builds available between 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6 and 98aa3624be70462d6a25ed5544333e3df62f4c66 within last 167 days

Even with the change in place, one would still probably want to paste the output into the github commend field and then look at the Preview to click through the issues, but at least this allows to filter out the obviously irrelevant ones easier.

Also expand rollup description to show individual rolled up prs

apiraino · 2025-05-05T09:54:55Z

This is an interesting addition. Just like you I also always inspect the rollup content manually.

Just one thought. The new output is now (understandably) verbose. Do you think it can be compressed with some clever markdown tricks?

Here an example, it's half-baked and does not render great (the Markdown nested syntax is very sensitive to spaces):

PRs in range:

#120748 (Rollup of 13 pull requests) by Nadrieril
PRs in rollup
- #110482 (Add armv8r-none-eabihf target for the Cortex-R52.)
- #119162 (Add unstable -Z direct-access-external-data cmdline flag for rustc)
- #120302 (various const interning cleanups)
- #120455 ( Add FileCheck annotations to MIR-opt SROA tests)
- #120470 (Mark "unused binding" suggestion as maybe incorrect)
- #120479 (Suggest turning if let into irrefutable let if appropriate)
- #120564 (coverage: Split out counter increment sites from BCB node/edge counters)
- #120633 (pattern_analysis: gather up place-relevant info)
- #120664 (Add parallel rustc ui tests)
- #120726 (Don't use bashism in checktools.sh)
- #120733 (MirPass: make name more const)
- #120735 (Remove some unchecked_claim_error_was_emitted calls)
- #120746 (Record coroutine kind in coroutine generics)
#120381 (Reconstify Add) by fee1-dead-contrib
#120521 (Make NonZero constructors generic.) by reitermarkus
#120558 (Stop bailing out from compilation just because there were incoherent traits) by oli-obk
#120579 (Prevent running some code if it is already in the map) by GuillaumeGomez
#120550 (Continue to borrowck even if there were previous errors) by oli-obk
#120767 (Rollup of 9 pull requests) by matthiaskrgr
PRs in rollup
- #119592 (resolve: Unload speculatively resolved crates before freezing cstore)
- #120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
- #120206 (hir: Make sure all HirIds have corresponding HIR Nodes)
- #120214 (match lowering: consistently lower bindings deepest-first)
- #120688 (GVN: also turn moves into copies with projections)
- #120702 (docs: also check the inline stmt during redundant link check)
- #120727 (exhaustiveness: Prefer "0..MAX not covered" to "_ not covered")
- #120734 (Add SubdiagnosticMessageOp as a trait alias.)
- #120739 (improve pretty printing for associated items in trait objects)
#120544 (Introduce enter_forall to supercede instantiate_binder_with_placeholders) by BoxyUwU
#120807 (Rollup of 9 pull requests) by matthiaskrgr
PRs in rollup
- #120590 (Remove unused args from functions)
- #120750 (No need to take ImplTraitContext by ref)
- #120769 (make future diffs minimal)
- #120772 (Remove myself from review rotation.)
- #120775 (Make min_exhaustive_patterns match exhaustive_patterns better)
- #120778 (Deduplicate tcx.instance_mir(instance) calls in try_instance_mir)
- #120782 (Fix mir pass ICE in the presence of other errors)
- #120783 (Add release note for new ambiguous_wide_pointer_comparisons lint)
- #120801 (Avoid ICE in drop recursion check in case of invalid drop impls)

ERROR: no CI builds available between 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6 and 98aa3624be70462d6a25ed5544333e3df62f4c66 within last 167 days

moxian · 2025-05-05T10:25:36Z

No, I don't think so. Three reasons:

rolled up PRs are not any less important or relevant when bisecting, so hiding those (and not others) feels odd and arbitrary
for very silly reasons I feel bad about "needlessly tagging innocent PRs", so when leaving a comment on the issue, I tend to post the bisection results inside a code block to prevent hotlinking. Like I did in the OP here. That would look bad with collapsibles. (But I can accept that this is a me issue and should be dismissed)
Similarly, this would also look bad in console (but we're already outputting markdown in console in other cases)

If PR list looks to long, then it's better to hide the entirety of it (and but just parts) under collapsible, IMO. I wouldn't mind adding those collapsible decorations for the entire list in the output by default, if we think it's a good idea.

apiraino · 2025-05-05T14:17:51Z

I don't feel too strongly about my suggestion but I think some form of collapsible could help reducing the clutter for visitors not interested in the bisection.

I also totally agree this bisection blurb should avoid tagging PRs in general, that's needless background noise across the repository. I use a github trick to avoid backlinks:

[https://togithub.com/rust/pull/123456](#123456)

Note the "togithub.com" domain. Could it be a solution to post backlinks quietly?

In any case thanks for submitting your idea (I like it!)

Use PR descriptions when presenting affected pr list.

e9691a0

Also expand rollup description to show individual rolled up prs

moxian marked this pull request as draft May 6, 2025 04:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use actual PR summaries instead of bors placeholders when listing them #379

Use actual PR summaries instead of bors placeholders when listing them #379

moxian commented May 3, 2025 •

edited

Loading

apiraino commented May 5, 2025

moxian commented May 5, 2025 •

edited

Loading

apiraino commented May 5, 2025

Use actual PR summaries instead of bors placeholders when listing them #379

Are you sure you want to change the base?

Use actual PR summaries instead of bors placeholders when listing them #379

Conversation

moxian commented May 3, 2025 • edited Loading

apiraino commented May 5, 2025

moxian commented May 5, 2025 • edited Loading

apiraino commented May 5, 2025

moxian commented May 3, 2025 •

edited

Loading

moxian commented May 5, 2025 •

edited

Loading