Various refactors to the LTO handling code #143388
Conversation
Most uses of it either contain a fat or thin lto module. Only WorkItem::LTO could contain both, but splitting that enum variant doesn't complicate things much.
As opposed to sending a message to the coordinator thread.
This will allow merging all fat LTO steps together. In addition it reduces the amount of work done on the coordinator thread without jobserver token.
|
rustbot has assigned @compiler-errors. Use |
rustbot
added
A-LLVM
|
Some changes occurred in compiler/rustc_codegen_gcc Some changes occurred in compiler/rustc_codegen_ssa Some changes occurred in compiler/rustc_codegen_llvm/src/builder/autodiff.rs cc @ZuseZ4 |
| autodiff: autodiff_items.clone(), | ||
| }, | ||
| 0, | ||
| )); | ||
| if cgcx.parallel { | ||
| helper.request_token(); | ||
| } |
There was a problem hiding this comment.
I'm not sure if this is necessary for fat LTO given that only a single thread will be doing LTO work and we have the implicit token anyway. It is pre-existing though and the token should be immediately yielded back to the jobserver if we indeed don't actually use it.
There was a problem hiding this comment.
My latest cleanup pr removes the clone, but you can land this pr first, as you say it didn't really hurt.
There was a problem hiding this comment.
I was actually talking about the helper.request_token() call.
In particular reducing the sharing of code paths between fat and thin-LTO and making the fat LTO implementation more self-contained. This also moves some autodiff handling out of cg_ssa into cg_llvm given that Enzyme only works with LLVM anyway and an implementation for another backend may do things entirely differently. This will also make it a bit easier to split LTO handling out of the coordinator thread main loop into a separate loop, which should reduce the complexity of the coordinator thread.