Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update utils #40

Merged
merged 1 commit into from
Feb 2, 2025
Merged

Update utils #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
199 changes: 101 additions & 98 deletions .github/workflows/continuous-integration.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ on:
push:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

env:
OMEGAUPUSER: ofmi
OMEGAUP_API_TOKEN: ${{ secrets.OMEGAUP_API_TOKEN }}
Expand All @@ -16,102 +20,101 @@ env:
jobs:
build-test-deploy:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Set base commit (pull request)
run: |
echo "GITHUB_BASE_COMMIT=${{ github.event.pull_request.base.sha }}" >> $GITHUB_ENV
echo "GITHUB_CURRENT_COMMIT=${{ github.event.pull_request.head.sha }}" >> $GITHUB_ENV
if: github.ref != 'refs/heads/main'

- name: Set base commit (main)
run: |
echo "GITHUB_BASE_COMMIT=${{ github.event.before }}" >> $GITHUB_ENV
echo "GITHUB_CURRENT_COMMIT=${{ github.event.after }}" >> $GITHUB_ENV
if: github.ref == 'refs/heads/main'

- name: Set up environment
run: |
git config --global core.quotePath false
echo "PIPENV_PIPFILE=$(pwd)/utils/Pipfile" >> $GITHUB_ENV
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: '3.8'

- uses: actions/checkout@v2
with:
ref: ${{ env.GITHUB_CURRENT_COMMIT }}
fetch-depth: 0
submodules: true

- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
(cd utils && sudo pip install pipenv && pipenv install)
- name: Set up the CI flags.
run: |
# When the utils/ directory is changed, run all tests to avoid
# regressions. Exit status of 0 means "unmodified".
if git diff-tree --quiet ${{ env.GITHUB_BASE_COMMIT }} ${{ env.GITHUB_CURRENT_COMMIT }} -- utils/; then
echo "CI_FLAGS=--ci" >> $GITHUB_ENV
else
echo "CI_FLAGS=--ci --all" >> $GITHUB_ENV
fi
- name: Download omegaUp-ci container
run: |
docker login https://docker.pkg.github.com -u ${{ github.actor }} -p ${{ github.token }}
python3 ./utils/runtests.py ${{ env.CI_FLAGS }} --only-pull-image
- name: Run tests
run: python3 ./utils/runtests.py ${{ env.CI_FLAGS }}

- name: Generate pngs/testplan
run: python3 ./utils/generateresources.py --generate=png,testplan ${{ env.CI_FLAGS }}
if: github.ref == 'refs/heads/main'

- name: Deploy to omegaUp
run: pipenv run python3 ./utils/upload.py --ci --verbose # Don't use CI_FLAGS to avoid deploying all problems.
if: github.ref == 'refs/heads/main'

- name: Push to public branch
if: github.ref == 'refs/heads/main'
run: |
shopt -s extglob
git config --global user.name "${{ env.GIT_USERNAME }}"
git config --global user.email "${{ env.GIT_USERNAME }}@users.noreply.github.com"
git add -f !(results)
git commit --allow-empty -m "Generated files from $GITHUB_SHA"
TMP_COMMIT=$(git rev-parse HEAD)
git checkout public
git merge --no-commit --no-ff -X theirs $TMP_COMMIT
git commit --allow-empty -m "Auto deployed from $GITHUB_SHA"
git push origin public

- name: Zip logs
if: ${{ always() }}
run: |
# actions/upload-artifacts action upload each and every log file
# individually through one API call. This is extremely slow (~2
# minutes) due to there being thousands of little files. Instead, .zip
# them all by hand and let the upload action just upload the .zip. Yes,
# this causes there to be a .zip within a .zip, but the billable
# minutes savings are worth it.
if [[ -d results/ && "$(find results/ -type f)" != "" ]]; then
(cd results && zip -r ../results.zip .)
fi

- name: Upload logs
if: ${{ always() }}
uses: actions/upload-artifact@v2
with:
name: omegaUp-CI-logs
path: results.zip

- name: Notify Slack on failure
uses: 8398a7/action-slack@v3
with:
status: ${{ job.status }}
fields: repo,message,commit,author,action,eventName,ref,workflow
env:
GITHUB_TOKEN: ${{ github.token }}
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
if: ${{ failure() && github.ref == 'refs/heads/main' }}
- name: Set base commit (pull request)
run: |
echo "GITHUB_BASE_COMMIT=${{ github.event.pull_request.base.sha }}" >> $GITHUB_ENV
echo "GITHUB_CURRENT_COMMIT=${{ github.event.pull_request.head.sha }}" >> $GITHUB_ENV
if: github.ref != 'refs/heads/main'

- name: Set base commit (main)
run: |
echo "GITHUB_BASE_COMMIT=${{ github.event.before }}" >> $GITHUB_ENV
echo "GITHUB_CURRENT_COMMIT=${{ github.event.after }}" >> $GITHUB_ENV
if: github.ref == 'refs/heads/main'

- name: Set up environment
run: |
git config --global core.quotePath false
echo "PIPENV_PIPFILE=$(pwd)/utils/Pipfile" >> $GITHUB_ENV

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.8"

- uses: actions/checkout@v2
with:
ref: ${{ env.GITHUB_CURRENT_COMMIT }}
fetch-depth: 0
submodules: true

- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
(cd utils && sudo pip install pipenv && pipenv install)

- name: Set up the CI flags.
run: |
# When the utils/ directory is changed, run all tests to avoid
# regressions. Exit status of 0 means "unmodified".
if git diff-tree --quiet ${{ env.GITHUB_BASE_COMMIT }} ${{ env.GITHUB_CURRENT_COMMIT }} -- utils/; then
echo "CI_FLAGS=--ci" >> $GITHUB_ENV
else
echo "CI_FLAGS=--ci --all" >> $GITHUB_ENV
fi

- name: Download omegaUp-ci container
run: |
docker login https://docker.pkg.github.com -u ${{ github.actor }} -p ${{ github.token }}
python3 ./utils/runtests.py ${{ env.CI_FLAGS }} --only-pull-image

- name: Run tests
run: python3 ./utils/runtests.py ${{ env.CI_FLAGS }}

- name: Generate pngs/testplan
run: python3 ./utils/generateresources.py --generate=png,testplan ${{ env.CI_FLAGS }}
if: github.ref == 'refs/heads/main'

- name: Deploy problems to omegaUp
run: pipenv run python3 ./utils/uploadproblems.py --ci --verbose --timeout=600 # Don't use CI_FLAGS to avoid deploying all problems.
if: github.ref == 'refs/heads/main'

- name: Deploy contests to omegaUp
run: pipenv run python3 ./utils/uploadcontests.py --ci --verbose # Don't use CI_FLAGS to avoid deploying all contests.
if: github.ref == 'refs/heads/main'

- name: Push to public branch
if: github.ref == 'refs/heads/main'
run: |
shopt -s extglob
git config --global user.name "${{ env.GIT_USERNAME }}"
git config --global user.email "${{ env.GIT_USERNAME }}@users.noreply.github.com"
git add -f !(results)
git commit --allow-empty -m "Generated files from $GITHUB_SHA"
TMP_COMMIT=$(git rev-parse HEAD)
git checkout public
git merge --no-commit --no-ff -X theirs $TMP_COMMIT
git commit --allow-empty -m "Auto deployed from $GITHUB_SHA"
git push origin public

- name: Zip logs
if: ${{ github.ref == 'refs/heads/main' || failure() }}
run: |
# actions/upload-artifacts action upload each and every log file
# individually through one API call. This is extremely slow (~2
# minutes) due to there being thousands of little files. Instead, .zip
# them all by hand and let the upload action just upload the .zip. Yes,
# this causes there to be a .zip within a .zip, but the billable
# minutes savings are worth it.
if [[ -d results/ && "$(find results/ -type f)" != "" ]]; then
(cd results && zip -r ../results.zip .)
fi

- name: Upload logs
if: ${{ github.ref == 'refs/heads/main' || failure() }}
uses: actions/upload-artifact@v4
with:
name: omegaUp-CI-logs
path: results.zip
2 changes: 1 addition & 1 deletion utils
Open in desktop
Submodule utils updated 11 files
+1 −1 .github/workflows/ci.yaml
+3 −8 Pipfile
+217 −85 Pipfile.lock
+3 −3 container.py
+311 −0 contests.py
+2 −2 generateresources.py
+5 −31 problems.py
+35 −0 repository.py
+2 −1 runtests.py
+81 −0 uploadcontests.py
+27 −9 uploadproblems.py
Loading