-
Notifications
You must be signed in to change notification settings - Fork 258
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(ci): run tarpaulin in release mode #4139
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #4139 +/- ##
==========================================
- Coverage 84.72% 78.00% -6.73%
==========================================
Files 269 268 -1
Lines 28779 24773 -4006
==========================================
- Hits 24384 19325 -5059
- Misses 4395 5448 +1053 ☔ View full report in Codecov by Sentry. |
Results of the experiment:
Since the build times are not fair comparisons (this PR could only have non-incremental build times, while for the baseline we get incremental compile times, I'm curious if others would like to enable the O1 level on main for a day or two, compare how it fares in terms of incremental compile times, and revert it if it's definitely not an improvement? cc @poljar |
Did you measure this locally or on CI? If we enable this on the CI how do we measure and compare this to our previous way, surely not via eyeballing? |
Eyeballing the wall clock times reported in the log statements 👀 Even if not precise at the very second, these give us the right orders of magnitude IMO. What I'd really want is likely to get an idea of the cost of an incremental build, and that could be done locally if someone were very bored… |
Ran some things during launch, to have a rough idea of how this would behave on my machine, in terms of compile times at least:
So, while this doesn't tell us what kind of compile-time slowdown would be caused on CI (especially since my machine has lots of CPU cores, and I suspect CI doesn't), it tells us a rebuild might be between > I'd still be inclined to try and merge this in production for a short while (a few days), rather than making back-of-the-envelope calculation like this, if others are interested. Otherwise I think it's ok to close this PR, there are other build-time optimizations to chase. |
This is a small experiment: it turns out compiling is relatively quick, on a PR I just did (3 minutes), but running the tarpaulin build took around 8 minutes. I'm wondering if increasing the compile times and lowering the run time is worth it, hence this experiment.
4f90b01
to
1659038
Compare
We chatted briefly about this, and we think it's an easy revert in case it's a regression. Let's find out :) |
A NEW RACE CONDITION APPEARS!
|
Because of (2), I think we should close this. It's possible that enabling O1 enables some extra inlining that confuses the codecov instrumentation, thus losing track of locations in the coverage. It's a bit of an unfortunate outcome, but at least we learned a thing 🥲 |
This is a small experiment: it turns out compiling is relatively quick, on a PR I just did (3 minutes), but running the tarpaulin build took around 8 minutes. I'm wondering if increasing the compile times and lowering the run time is worth it, hence this experiment.