Incremental cache in Docker cluster #555
AhmedMozaly
started this conversation in
RFCs
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Although I've just created a PoC PR, but better to have the discussion here:
When building a docker image, There are two types of cache content that Docker maintain between builds:
/root/.npm
)Both cache content are served/handled by the local Docker daemon, this means that an image built on specific Docker host. need to be built on the same host each time to take full advantage of caching capabilities
This PR enables Nixpacks to work more smoothly in a cluster setup (Multiple Docker nodes accessing cache remotely)
After build command runs (for example
npm run build
) the directories need to be cached (like/root/.npm
) will be archived with tar and sent to a small HTTP server that has been started by Nixpacks to receive these filesThese tar files -if available- will be copied back to any newer image build. It will extracted to the right location inside the image to help build tools find out the files needed for the build without the needed to re-download it.
Plan:
Feedback is welcomed!
Beta Was this translation helpful? Give feedback.
All reactions