-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Used cached Rust registry for conda builds #1167
Conversation
Codecov Report
❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more. @@ Coverage Diff @@
## main #1167 +/- ##
==========================================
+ Coverage 81.46% 81.58% +0.11%
==========================================
Files 78 78
Lines 4403 4403
Branches 798 798
==========================================
+ Hits 3587 3592 +5
+ Misses 640 631 -9
- Partials 176 180 +4 see 1 file with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
if [[ -v GITHUB_WORKSPACE ]]; then | ||
if [[ ! -d "$GITHUB_WORKSPACE/dask_planner/target" ]]; then | ||
mkdir "$GITHUB_WORKSPACE/dask_planner/target" | ||
fi | ||
ln -s $GITHUB_WORKSPACE/dask_planner/target $CONDA_PREFIX/../work/dask_planner/target | ||
fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding this to the script makes it so we should now be sharing work between builds on the same job (i.e. if we're building all packages in one conda build
command intermediate compilation only needs to be done once); however, since each conda build
command generates its own unique directory (e.g. /usr/share/miniconda3/envs/test/conda-bld/dask-sql_1685500788622
), this work can't be shared between jobs because the source path is used as part of detecting a cache hit.
Going to explore if we can either:
- Configure
conda build
to use a reproducible work directory on every run - Configure the Rust caching to cache on relative path versus absolute
cc @ayushdg in case you've played around with the Rust caching at all
Hopefully this should speed up conda builds on PRs and commits