Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a way to cache the complete layers for copy work #2419

Open
cowcomic opened this issue Sep 6, 2024 · 2 comments
Open

Is there a way to cache the complete layers for copy work #2419

cowcomic opened this issue Sep 6, 2024 · 2 comments

Comments

@cowcomic
Copy link

cowcomic commented Sep 6, 2024

With a poor network the copy work often failed A large image needs to download all the layers every time and it will fail every time.
I need a way to cache the downloaded layer so that I don’t have to download it again when I copy it again after failure.

@mtrmac
Copy link
Collaborator

mtrmac commented Sep 6, 2024

Thanks for reaching out.

A skopeo copy [--all --preserve-digests] docker://… dir: will create a local copy, which can then be pushed elsewhere. But for that, the whole image download needs to succeed, because a write to dir: wipes the previous contents.

Alternatively, you can run a temporary registry on localhost (maybe in a container); that one does work a layer at a time, and on a failure, restarts the copy from the failed layer (from start).

In recent versions, downloads from registries have a limited resume logic, so rare network interruptions of specific kinds don’t terminate the whole layer operation. But uploads do need to happen whole layers at a time, so if an upload is interrupted, it can only be restarted from scratch.

@cowcomic
Copy link
Author

cowcomic commented Sep 8, 2024

OK,i will try to use registry as a hub

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants