Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schedule CSV Generation #10

Merged
merged 8 commits into from
Feb 1, 2024
Merged

Schedule CSV Generation #10

merged 8 commits into from
Feb 1, 2024

Conversation

ibrahimjaved12
Copy link
Collaborator

@ibrahimjaved12 ibrahimjaved12 commented Jan 30, 2024

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/3326

Description (What does it do?)

This PR adds a Github Action which creates a weekly Release with the newly created CSV file.
If this step fails: poetry run python -m ocw_oer_export.cli --create_csv --output_path="./private/output/ocw_oer_export.csv"

Then it creates a Slack alert for the notifications-ocw-oer-export channel, such as this:
image
Although the event would be schedule or push, not pull_request (that was for testing purposes).

All the releases would be found here: https://github.com/mitodl/ocw_oer_export/releases
The latest release would be found on this URL: https://github.com/mitodl/ocw_oer_export/releases/tag/latest

The release tags follow the semversioning pattern:
vx.y.z-csv (where z would be incremented every week).
Eg. v1.0.1-csv, v1.0.2-csv.

How can this be tested?

Feel free to produce errors on this PR as commit and then check for Slack notifications. You can also check from the error I produced (from the commits) and its Slack notification (Check the existing Actions on the repo)

The steps to run the program are same as before, but that won't test the Github Actions.

  1. Checkout to this branch
  2. Build the container:
docker compose build
  1. Start the container:
docker compose run --rm app
  1. To generate a JSON file containing complete API data:
docker compose run --rm app --create_json
  1. To create a CSV file from the local JSON file:
docker compose run --rm app --create_csv --source=json
  1. To run unit tests:
docker run --rm ocw_oer_export python -m unittest discover

@ibrahimjaved12 ibrahimjaved12 marked this pull request as ready for review January 30, 2024 14:08
@ibrahimjaved12 ibrahimjaved12 added the Needs Review An open Pull Request that is ready for review label Jan 30, 2024
@gumaerc gumaerc self-assigned this Jan 30, 2024
@ibrahimjaved12
Copy link
Collaborator Author

@gumaerc The README.md will also have to be updated after this. I'll do it afterwards we establish that we're good to go with the current approach. I might update it in the other, trivial OCW OER Export PR I'm working on - to reduce the number of PRs to review.

Copy link

@gumaerc gumaerc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

I tested this locally using [act](https://github.com/nektos/act) and it successfully generated a CSV. It failed when it went to create the release with:

| Creating release: v1.0.3-csv
| /var/run/act/workflow/9: line 3: gh: command not found

... but I think it's just an issue with act itself.

@ibrahimjaved12 ibrahimjaved12 merged commit 1c01629 into main Feb 1, 2024
2 checks passed
@ibrahimjaved12 ibrahimjaved12 deleted the schedule-csv-generation branch February 1, 2024 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review An open Pull Request that is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants