Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix updates from repodata #1

Merged
merged 16 commits into from
Jun 11, 2024
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# GitHub syntax highlighting
pixi.lock linguist-language=YAML

10 changes: 10 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
groups:
github-actions:
patterns:
- "*"
77 changes: 61 additions & 16 deletions .github/workflows/bootstrap.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,17 @@
name: Update database

on: [push, workflow_dispatch]
on:
push:
branches:
- main

env:
DB_PATH: ${{ github.workspace }}/path_to_artifacts.db
PYTHONUNBUFFERED: 1
pull_request:

workflow_dispatch:

schedule:
- cron: "0 5 * * 1" # Mondays @ 5am
- cron: "0 1 1 * *" # First day of the month @ 1am

concurrency:
# Concurrency group that uses the workflow name and PR number if available
Expand All @@ -15,6 +22,13 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

permissions:
contents: write

env:
DB_PATH: path_to_artifacts.db
PYTHONUNBUFFERED: 1

jobs:
update:
runs-on: ubuntu-latest
Expand All @@ -34,34 +48,65 @@ jobs:
tar xf path_to_artifacts.tar.zst
rm path_to_artifacts.tar.zst

- uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install dependencies
run: pip install -r requirements.txt
- uses: prefix-dev/[email protected]

- name: Update database with most recent repodata
run: |
set -x
ls -alh *.db
python conda_forge_paths/path_to_artifacts_db.py update-from-repodata
pixi run python conda_forge_paths/path_to_artifacts_db.py update-from-repodata
ls -alh *.db

- name: Update FTS index
run: |
set -x
ls -alh *.db
python conda_forge_paths/path_to_artifacts_db.py fts
pixi run python conda_forge_paths/path_to_artifacts_db.py fts
ls -alh *.db

- name: Get current time
uses: josStorer/get-current-time@v2
id: current-time
with:
format: "YYYY-MM-DD"
utcOffset: "+00:00"

- name: Compress DB file
run: |
ZSTD_NBTHREADS=4 ZSTD_CLEVEL=19 tar --zstd -cf path_to_artifacts.tar.zstd $DB_PATH
ZSTD_NBTHREADS=$(nproc) ZSTD_CLEVEL=19 tar --zstd -cf path_to_artifacts.tar.zstd $DB_PATH
ls -alh path_to_artifacts.*

- name: "Upload Artifact"
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
with:
name: database
path: ${{ env.DB_PATH }}.tar.zstd
path: ${{ github.workspace }}/${{ env.DB_PATH }}.tar.zstd
retention-days: 60
compression-level: 0

- name: Publish weekly
uses: softprops/action-gh-release@v2
if: github.event_name == 'schedule'
with:
draft: false
name: "Latest Release"
tag_name: "latest"
files: path_to_artifacts.tar.zstd
fail_on_unmatched_files: true
body: |
Weekly snapshot of the conda-forge path-to-artifacts database.
Last updated on ${{ steps.current-time.outputs.formattedTime }}.
See the README for more information.

- name: Publish monthly or manually
uses: softprops/action-gh-release@v2
if: ( github.event_name == 'schedule' && endsWith(steps.current-time.outputs.formattedTime, '-01') ) || github.event_name == 'workflow_dispatch'
with:
draft: false
name: ${{ steps.current-time.outputs.formattedTime }}
tag_name: ${{ steps.current-time.outputs.formattedTime }}
files: path_to_artifacts.tar.zstd
fail_on_unmatched_files: true
body: |
${{ steps.current-time.outputs.formattedTime }} snapshot of the conda-forge path-to-artifacts database.
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -160,5 +160,11 @@ cython_debug/
#.idea/

*.db
*.db-journal
*.zst
.repodata_cache/

# pixi environments
.pixi
*.egg-info

Loading
Loading