Skip to content

src: add an option to make compile cache path relative #58797

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Aditi-1400
Copy link
Contributor

@Aditi-1400 Aditi-1400 commented Jun 23, 2025

Adds an option (NODE_COMPILE_CACHE_RELATIVE_PATH) for the built-in compile cache to encode the hashes with relative file paths. On enabling the option,
the source directory along with cache directory can be bundled and moved, and the cache continues to work.

When enabled, paths encoded in hash are relative to compile cache directory.

Fixes: #58755

Thanks @joyeecheung for all the help :)

Adds an option (NODE_COMPILE_CACHE_RELATIVE_PATH) for
the built-in compile cache to encode the hashes with
relative file paths. On enabling the option,
the source directory along with cache directory can be
bundled and moved, and the cache continues to work.

When enabled, paths encoded in hash are relative to
compile cache directory.
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jun 23, 2025
Copy link

codecov bot commented Jun 23, 2025

Codecov Report

Attention: Patch coverage is 67.79661% with 19 lines in your changes missing coverage. Please review.

Project coverage is 90.09%. Comparing base (85e8cc6) to head (0d4fb87).
Report is 58 commits behind head on main.

Files with missing lines Patch % Lines
src/compile_cache.cc 64.00% 9 Missing and 9 partials ⚠️
src/env.cc 88.88% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #58797      +/-   ##
==========================================
- Coverage   90.16%   90.09%   -0.07%     
==========================================
  Files         636      640       +4     
  Lines      188061   188341     +280     
  Branches    36899    36927      +28     
==========================================
+ Hits       169561   169692     +131     
- Misses      11245    11363     +118     
- Partials     7255     7286      +31     
Files with missing lines Coverage Δ
src/compile_cache.h 100.00% <ø> (ø)
src/node_file.h 78.43% <ø> (ø)
src/env.cc 80.98% <88.88%> (+0.16%) ⬆️
src/compile_cache.cc 78.18% <64.00%> (-2.60%) ⬇️

... and 76 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Member

@joyeecheung joyeecheung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some integration to module.enableCompileCache() as well? I think having it in options bag when the first argument is an object might work well enough. The path can be options.path in that case.

@@ -3181,6 +3181,10 @@ added: v22.1.0
Enable the [module compile cache][] for the Node.js instance. See the documentation of
[module compile cache][] for details.

### `NODE_COMPILE_CACHE_RELATIVE_PATH=0`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if "relative" is more of an implementation detail, what we really want to allow is "portable" compile cache (i.e. you can move the code and the compile cache to a different base directory, the cache will still be hit as long as they maintain the same directory structure) So maybe NODE_COMPILE_CACHE_PORTABLE=0 would be more intuitive and we can explain about the directory structure with an example in the docs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, please add some documentation to the Module compile cache section in module.md as well.

if (use_relative_ && IsAbsoluteFilePath(file_path)) {
// Normalise the paths to ensure they are consistent.
std::string normalised_file_path = NormalisePath(file_path);
std::string normalised_cache_dir =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something we can just compute once and store in the CompileCacheHandler instance. This is currently being repeatedly computed for every path match even though it's the same every time.

// This tests NODE_COMPILE_CACHE works with the NODE_COMPILE_CACHE_RELATIVE_PATH
// environment variable.

require('../common');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some tests for module.getCompileCacheDir() as well? I think it should return a relative path (from process.cwd()?) in that case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support relative compile cache
3 participants