Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tool for gzipping a workspace on backend #569

Open
bloodearnest opened this issue Feb 1, 2023 · 1 comment
Open

Tool for gzipping a workspace on backend #569

bloodearnest opened this issue Feb 1, 2023 · 1 comment
Assignees

Comments

@bloodearnest
Copy link
Member

bloodearnest commented Feb 1, 2023

We have various large workspace with many uncompressed csv files

We want users to switch to csv.gz, but we don't want them to have to re-run all the jobs because a) time and b) it wouldn't actually remove the existing files

Instead, we should write a new jobrunner.cli tool that, for a given workspace

for each .csv in the workspace:

  • gzips the file
  • updates the database for any actions that produced that file to point to the .gz version instead.

This would then allow the user to switch to csf.gz in their project.yaml and code, and everything should just continue as it had been that way all along

@bloodearnest bloodearnest self-assigned this Feb 1, 2023
@bloodearnest
Copy link
Member Author

Bristol are going to create new workspaces that use compressed files, and archive the older workspaces, so we can then compress and archive them.

So this tool might not be needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant