Skip to content

Commit

Permalink
Update to Node.js 20 (#1)
Browse files Browse the repository at this point in the history
* Minor improvements

* Test functionality

* (Automated) Updated iTunes Country codes

* Resolve lint

* Resolve lint

* Re-add release block

* New line

* Remove legacy lint-action

---------

Co-authored-by: jcoester <[email protected]>
  • Loading branch information
jcoester and jcoester authored Apr 4, 2024
1 parent 5325807 commit 5aa56f4
Show file tree
Hide file tree
Showing 7 changed files with 39 additions and 20 deletions.
43 changes: 31 additions & 12 deletions .github/workflows/fetch-itunes-country-codes.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,17 @@ name: Fetch iTunes Country codes

on:
push:
branches: [ "main" ]
branches:
- main
- dev
paths-ignore:
- '**/README.md'
pull_request:
branches: [ "main" ]
branches:
- main
- dev
schedule:
- cron: "0 23 * * 1-5" # 23:00 UTC Monday-Friday
- cron: "59 23 * * 1-5" # 23:59 UTC every Monday to Friday

permissions:
contents: write
Expand All @@ -19,57 +23,72 @@ jobs:
strategy:
matrix:
# split task into parallel chunks to avoid iTunes Search API throttling
chunks: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
chunks: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
env:
total_chunks: 12
steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install requests
- name: Fetch iTunes Country codes
run: python script/fetch_itunes_country_codes.py $total_chunks ${{matrix.chunks}} > ./c${{matrix.chunks}}.json
- uses: actions/upload-artifact@v4
run: python script/fetch_itunes_country_codes.py $total_chunks ${{matrix.chunks}} > ./c_${{matrix.chunks}}.json

- name: Upload chunks
uses: actions/upload-artifact@v4
with:
name: chunks-${{ matrix.chunks }}
path: ./c${{matrix.chunks}}.json
path: ./c_${{matrix.chunks}}.json

combine:
runs-on: ubuntu-latest
needs: [fetch-chunks]
steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install pandas
- name: Get current date
id: date
run: echo "date=$(date +'%Y-%b-%d')" >> $GITHUB_OUTPUT
- name: Download all workflow run artifacts

- name: Download all fetched chunks
uses: actions/download-artifact@v4
with:
pattern: chunks-*
merge-multiple: true

- name: Combine chunks
id: changes
run: |
echo "CHANGELOG<<EOF" >> $GITHUB_ENV
python script/combine_chunks.py >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
- uses: stefanzweifel/git-auto-commit-action@v5
- name: Auto-commit changes
uses: stefanzweifel/git-auto-commit-action@v5
with:
commit_message: (Automated) Updated iTunes Country codes
commit_message: "(Automated) Updated iTunes Country codes"
file_pattern: 'itunes_country_codes.json'

- name: Release
uses: softprops/action-gh-release@v2
if: env.CHANGELOG != 'identical'
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ on:
push:
branches:
- main
- dev
pull_request:
branches:
- main
- dev

permissions:
checks: write
Expand All @@ -21,7 +23,7 @@ jobs:
- name: Check out Git repository
uses: actions/checkout@v4

- name: Set up Python
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: "3.10"
Expand All @@ -31,5 +33,5 @@ jobs:

- name: Run linters
run: |
flake8 --max-line-length=100 .
black .
flake8 .
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@

List of all available iTunes Country codes used for iTunes Store, Search API and RSS Feed.

A GitHub Action runs every Mo-Fr at 23:00 UTC and publishes detected changes automatically.
A GitHub Action runs every Mo-Fr at 23:59 UTC and publishes detected changes automatically.

[Permalink to JSON file](itunes_country_codes.json) ([Raw](itunes_country_codes.json?raw=1))
6 changes: 2 additions & 4 deletions script/combine_chunks.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,7 @@
full = ""

# List all files in the directory
file_list = [
f for f in os.listdir(base_path) if f.startswith("c") and f.endswith(".json")
]
file_list = [f for f in os.listdir(base_path) if f.startswith("c_")]

# Combine chunk data
for file in file_list:
Expand All @@ -28,7 +26,7 @@

# 2. Load list of alpha-2 country names
df_countries = pd.read_csv(
"script/alpha2_countries.csv", encoding="utf8", keep_default_na=False
"script/data/alpha2_countries.csv", encoding="utf8", keep_default_na=False
).set_index("alpha2")
dict_countries = df_countries["country"].to_dict()

Expand Down
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion script/fetch_itunes_country_codes.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def split(alist, n):
def main():
# 1. Load list of all 676 available ISO 3166-1 alpha-2 codes
# From https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2
list_alpha2 = json.load(open("script/iso3166_1_alpha2.json"))
list_alpha2 = json.load(open("script/data/iso3166_1_alpha2.json"))
assert len(list_alpha2) == 676

# 2. Split into chunks to enable parallel execution
Expand Down

0 comments on commit 5aa56f4

Please sign in to comment.