Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Further save-daily-metrics.yml improvements #175

Open
1 task
Tracked by #128
e-belfer opened this issue Sep 17, 2024 · 0 comments
Open
1 task
Tracked by #128

Further save-daily-metrics.yml improvements #175

e-belfer opened this issue Sep 17, 2024 · 0 comments
Labels
github_actions Pull requests that update GitHub Actions code

Comments

@e-belfer
Copy link
Member

e-belfer commented Sep 17, 2024

Overview

What is the problem we're solving? For very simple items, this can be encapsulated in the success criteria.
The save_daily_metrics run fails if the file already exists. It will also not run the Kaggle archiver if the Github archiver fails. Let's fix this!

Success Criteria

How will we know that we're done?

  • If a file already exists, we catch the error rather than getting a "does not have storage.objects.delete access" error
  • Switch to using a requests.get() method with retry logic in all saved archives
  • Use a matrix to run each archiver simultaneously

Next steps

Preview Give feedback
@e-belfer e-belfer added the github_actions Pull requests that update GitHub Actions code label Sep 17, 2024
@e-belfer e-belfer moved this from New to Backlog in Catalyst Megaproject Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
github_actions Pull requests that update GitHub Actions code
Projects
Status: Backlog
Development

No branches or pull requests

1 participant