You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I manage millions of small files that needs to be concatenated and your tool works great for that.
In the case of gzipped source files, I needed to decompress the file before doing the concatenation based on the ContentEncoding header in S3 response.
If you're interested, I can submit a PR with the changes to handle automatically gzipped content (It's only 6 lines of code).
Maybe an arg can be added in MultipartUploadJob if you wan't to enable it manually.
Thanks for this lib.
The text was updated successfully, but these errors were encountered:
Hi,
I manage millions of small files that needs to be concatenated and your tool works great for that.
In the case of gzipped source files, I needed to decompress the file before doing the concatenation based on the
ContentEncoding
header in S3 response.If you're interested, I can submit a PR with the changes to handle automatically gzipped content (It's only 6 lines of code).
Maybe an arg can be added in
MultipartUploadJob
if you wan't to enable it manually.Thanks for this lib.
The text was updated successfully, but these errors were encountered: