-
Notifications
You must be signed in to change notification settings - Fork 377
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pkg/compression: new zstd variant zstd:chunked #1084
Commits on Jul 2, 2021
-
compression: change return type for zstdWriterWithLevel
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 48f39f6 - Browse repository at this point
Copy the full SHA 48f39f6View commit details -
compression: support generating compressor metadata
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for b167b4b - Browse repository at this point
Copy the full SHA b167b4bView commit details -
compression: let algorithms register a MIME type
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 7421a48 - Browse repository at this point
Copy the full SHA 7421a48View commit details -
pkg/compression: new zstd variant zstd:chunked
add a new custom variant of the zstd compression that permits to retrieve each file separately. The idea is based on CRFS and its stargz format for having seekable and indexable tarballs. One disadvantage of the stargz format is that a custom file is added to the tarball to store the layer metadata, and the metadata file is part of the image itself. Clients that are not aware of the stargz format will propagate the metadata file inside of the containers. The zstd compression supports embeddeding additional data as part of the stream that the zstd decompressor will ignore (skippable frame), so the issue above with CRFS can be solved directly within the zstd compression format. Beside this minor advantage, zstd is much faster and compresses better than gzip, so take this opportunity to push the zstd format further. The zstd compression is supported by the OCI image specs since August 2019: opencontainers/image-spec#788 and has been supported by containers/image since then. Clients that are not aware of the zstd:chunked format, won't notice any difference when handling a blob that uses the variant. Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 40848f4 - Browse repository at this point
Copy the full SHA 40848f4View commit details -
types: add interface for partial blob retrieval
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for e8ce5b8 - Browse repository at this point
Copy the full SHA e8ce5b8View commit details -
types: add interface for storing partial blob
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 05f70a3 - Browse repository at this point
Copy the full SHA 05f70a3View commit details -
docker: support partial blob retrieval
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 93477e6 - Browse repository at this point
Copy the full SHA 93477e6View commit details -
storage: support partial storage with zstd:chunked
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 67a4c99 - Browse repository at this point
Copy the full SHA 67a4c99View commit details -
copy: do not ignore errors on Close
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 3250f2d - Browse repository at this point
Copy the full SHA 3250f2dView commit details -
copy: use partial blob retrieval if possible
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 138ab62 - Browse repository at this point
Copy the full SHA 138ab62View commit details -
copy: provide progress bar for partial blobs
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for ee5014c - Browse repository at this point
Copy the full SHA ee5014cView commit details -
copy: make using partial blobs configurable
Signed-off-by: Giuseppe Scrivano <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for be50ba7 - Browse repository at this point
Copy the full SHA be50ba7View commit details