Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compress text output files #38

Open
wtraylor opened this issue May 15, 2021 · 0 comments
Open

Compress text output files #38

wtraylor opened this issue May 15, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@wtraylor
Copy link
Owner

The text output files can build up to substantial size. It would be convenient to have the option that they are automatically compressed when the simulation is complete.

  • Add TOML key output.text_tables.compression with the options "none" and "gzip".
  • If "gzip" is selected, each TSV output file gets compressed at the end of the simulation or on the fly. That creates for instance mass_density_per_hft.tsv.gz.
  • The demo_results.Rmd file checks if the output is compressed, and if so, uses R’s gzfile() function to decompress it on the fly. Compare this stackoverflow post.
  • As for a library, a wrapper around zlib seems to be the way to go. There are gzstream and zstr, which both expose C++11 output streams.
@wtraylor wtraylor added the enhancement New feature or request label May 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant