Perf ? compressionLevel without affecting lock file #4965
belgattitude
started this conversation in
Ideas
Replies: 1 comment 3 replies
-
The checksums are intended to make sure that, when running |
Beta Was this translation helpful? Give feedback.
3 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I've been interested about carbon emissions recently and started tuning yarn install on multiple repos (ie: gha yarn install with action/cache gist).
Intro
Right now I'm looking into compressionLevel and here's some results:
ie: projection 100 install/week with
compressionLevel=0
saves ♻️ ~53 minutes of CI/month fromcompressionLevel=mixed
(>25% speedup).Or locally (using taskset to emulate single core)
YARN_COMPRESSION_LEVEL=0 taskset -c 0 yarn install --immutable --mode=skip-build
YARN_COMPRESSION_LEVEL=mixed taskset -c 0 yarn install --immutable --mode=skip-build
Hyperfine/Taskset cmd
The benchmarks runs with node_module linker (pnp has issues with vercel/nft...) and are based on https://github.com/belgattitude/nextjs-monorepo-example (monorepo with nextjs, storybook, linters...)
For more info, please look at this PR belgattitude/nextjs-monorepo-example#2799. Very easy to reproduce.
To sum up the
compressionLevel:0
is very helpful option 🙏 👍Drawbacks
Currently the
compressionLevel:0
has some drawbacks. We can't set it differently locally (saves 100's of MB of disk storage) and on the CI. Cause it affect theyarn.lock
file (metadata and cheksums) and in the big picture (hardlinks-global, renovate bot, changesets ....) it will cause issues.Possible improvements
I'm wondering if there's a strong reason to store the cheksums in yarn.lock ? What if
./yarn/cache/tbd
,install-state
...That way:
Implementation ideas
Before going further, I'd like to gather some feedbacks from @arcanis or any yarn team member. I'm not sure of the feasibility.
Beta Was this translation helpful? Give feedback.
All reactions