-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Configure Compression Settings #18
Comments
Unfortunately this is hardcoded ATM - we need to modify this section to be dynamic. We should do this by adding a compression configuration node (maybe under the format node, or perhaps its own node 🤔) Thinking big picture, we want to satisfy the following requirements:
|
Thank you @crowemi for the fast response. |
ok. I was looking for a static filename rather than a timestamp, but thank you anyway. |
+1 for this. Current default option (gzip) isn't he best option for Parquet files:
|
Hi, i would like to work on this issue. Since there is no contribution guidelines, is there anything i should pay attention to ? |
I have tried target-s3 and I was able to change prefix and stream_name_path_override but inside the S3 folder I get a compressed ".json.gz". Instead, I need a file called "data.json" without compression.
The text was updated successfully, but these errors were encountered: