Skip to content

Commit

Permalink
Ensure PR workflows can run on all PRs (but still skip when they're n…
Browse files Browse the repository at this point in the history
…ot needed) (#2237)

Closes #2279

This adds a `paths-filter` step to all workflows that run on PRs so that
we can enable auto-merge, for more info about this read [enabling
auto-merge](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/automatically-merging-a-pull-request)!
In order to be able to have required checks, a workflow can't be
entirely skipped: see [Handling skipped but required
checks](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/troubleshooting-required-status-checks#handling-skipped-but-required-checks).

This also merges frontend CI workflows into one with multiple parallel
steps, which should speed things up a bit. It also upgrades node
versions to 20 and 22 across the board.
emma-sg authored Jan 14, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
1 parent c563b62 commit 318acaf
Showing 8 changed files with 469 additions and 256 deletions.
42 changes: 35 additions & 7 deletions .github/workflows/ansible-lint.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,42 @@
name: ansible-lint
on:
push:
paths:
- 'ansible/**'
pull_request:
paths:
- 'ansible/**'

on: [push, pull_request]

# Cancel in progress workflows on pull_requests.
# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
# In order to be able to have required checks, a workflow can't be entirely
# skipped: see https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/troubleshooting-required-status-checks#handling-skipped-but-required-checks
paths-filter:
name: "Changed files?"
runs-on: ubuntu-latest
outputs:
matches: ${{ steps.filter.outputs.matches }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2 # important, to fetch previous commit

# workaround for https://github.com/dorny/paths-filter/issues/240
- id: previous-sha
run: 'echo "sha=$(git rev-parse HEAD^1)" >> $GITHUB_OUTPUT'

- uses: dorny/paths-filter@v3
id: filter
with:
base: "${{ steps.previous-sha.outputs.sha }}"
filters: |
matches:
- 'ansible/**'
- '.github/workflows/ansible-lint.yaml'
build:
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
name: Ansible Lint
runs-on: ubuntu-latest

69 changes: 69 additions & 0 deletions .github/workflows/backend-lint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: Backend Lint + Type Check

on: [push, pull_request]

# Cancel in progress workflows on pull_requests.
# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
# In order to be able to have required checks, a workflow can't be entirely
# skipped: see https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/troubleshooting-required-status-checks#handling-skipped-but-required-checks
paths-filter:
name: "Changed files?"
runs-on: ubuntu-latest
outputs:
matches: ${{ steps.filter.outputs.matches }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2 # important, to fetch previous commit

# workaround for https://github.com/dorny/paths-filter/issues/240
- id: previous-sha
run: 'echo "sha=$(git rev-parse HEAD^1)" >> $GITHUB_OUTPUT'

- uses: dorny/paths-filter@v3
id: filter
with:
base: "${{ steps.previous-sha.outputs.sha }}"
filters: |
matches:
- 'backend/**'
- '.github/workflows/backend-lint.yaml'
unit-tests:
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Install dependencies
run: |
cd backend/
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r dev-requirements.txt
- name: Style Check
run: |
black --check backend/btrixcloud/
- name: Lint Check
run: |
cd backend/
pylint btrixcloud/
- name: Type Check
run: |
cd backend/
mypy --install-types --non-interactive --check-untyped-defs btrixcloud/
101 changes: 0 additions & 101 deletions .github/workflows/frontend-build-prepare.yaml

This file was deleted.

299 changes: 299 additions & 0 deletions .github/workflows/frontend-lint-test-build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,299 @@
name: Frontend
on:
pull_request

# Cancel in progress workflows on pull_requests.
# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
# In order to be able to have required checks, a workflow can't be entirely
# skipped: see https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/troubleshooting-required-status-checks#handling-skipped-but-required-checks
paths-filter:
name: "Changed files?"
runs-on: ubuntu-latest
outputs:
matches: ${{ steps.filter.outputs.matches }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2 # important, to fetch previous commit

# workaround for https://github.com/dorny/paths-filter/issues/240
- id: previous-sha
run: 'echo "sha=$(git rev-parse HEAD^1)" >> $GITHUB_OUTPUT'

- uses: dorny/paths-filter@v3
id: filter
with:
base: "${{ steps.previous-sha.outputs.sha }}"
filters: |
matches:
- 'frontend/src/**'
- 'frontend/*.json'
- 'frontend/*.js'
- 'frontend/*.ts'
- '.github/workflows/frontend-lint-test-build.yaml'
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
strategy:
matrix:
node: [20, 22]
steps:
# Setup:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}

- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: 'yarn'
cache-dependency-path: frontend/yarn.lock

- name: Get installed Playwright version
working-directory: frontend
id: playwright-version
run: |
yarn add @yarnpkg/lockfile
echo "PLAYWRIGHT_VERSION=$(node scripts/get-resolved-playwright-version.js)" >> $GITHUB_ENV
- name: Cache playwright binaries
uses: actions/cache@v4
id: playwright-cache
with:
path: |
~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ env.PLAYWRIGHT_VERSION }}

- name: Install dependencies
working-directory: frontend
env:
HUSKY: 0
run: yarn install --frozen-lockfile

- name: Install Playwright Browsers
working-directory: frontend
env:
HUSKY: 0
run: yarn playwright install --with-deps
if: steps.playwright-cache.outputs.cache-hit != 'true'

- name: Install Playwright Deps
working-directory: frontend
env:
HUSKY: 0
run: yarn playwright install-deps
if: steps.playwright-cache.outputs.cache-hit != 'true'

# Test:
- name: Unit tests
working-directory: frontend
run: yarn test

integration-tests:
name: Integration Tests
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
timeout-minutes: 60
runs-on: ubuntu-latest
strategy:
matrix:
node: [20, 22]
env:
E2E_USER_EMAIL: ${{ secrets.E2E_USER_EMAIL }}
E2E_USER_PASSWORD: ${{ secrets.E2E_USER_PASSWORD }}
API_BASE_URL: ${{ secrets.API_BASE_URL }}
working-directory: ./frontend
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: 'yarn'
cache-dependency-path: frontend/yarn.lock

- name: Get installed Playwright version
working-directory: frontend
id: playwright-version
run: |
yarn add @yarnpkg/lockfile
echo "PLAYWRIGHT_VERSION=$(node scripts/get-resolved-playwright-version.js)" >> $GITHUB_ENV
- name: Cache playwright binaries
uses: actions/cache@v4
id: playwright-cache
with:
path: |
~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ env.PLAYWRIGHT_VERSION }}

- name: Install dependencies
working-directory: frontend
run: yarn install --frozen-lockfile

- name: Install Playwright Browsers
working-directory: frontend
env:
HUSKY: 0
run: yarn playwright install --with-deps
if: steps.playwright-cache.outputs.cache-hit != 'true'

- name: Install Playwright Deps
working-directory: frontend
env:
HUSKY: 0
run: yarn playwright install-deps
if: steps.playwright-cache.outputs.cache-hit != 'true'

- name: Create env file
working-directory: frontend
run: |
touch .env
echo E2E_USER_EMAIL="${{ secrets.E2E_USER_EMAIL }}" >> .env
echo E2E_USER_PASSWORD="${{ secrets.E2E_USER_PASSWORD }}" >> .env
cat .env
- name: Build frontend
working-directory: frontend
run: yarn build
id: build-frontend

- name: Run Playwright tests
working-directory: frontend
run: yarn playwright test

- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report-node${{ matrix.node }}-${{github.sha}}
path: frontend/playwright-report/
retention-days: 30

lint:
name: Lint
runs-on: ubuntu-latest
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
strategy:
matrix:
node: [20, 22]
steps:
# Setup:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}

- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: 'yarn'
cache-dependency-path: frontend/yarn.lock

- name: Install dependencies
working-directory: frontend
env:
HUSKY: 0
run: yarn install --frozen-lockfile

# Lint:
- name: Lint
working-directory: frontend
run: yarn lint:check

localize:
name: Localize
runs-on: ubuntu-latest
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
strategy:
matrix:
node: [20, 22]
steps:
# Setup:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}

- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: 'yarn'
cache-dependency-path: frontend/yarn.lock

- name: Install dependencies
working-directory: frontend
env:
HUSKY: 0
run: yarn install --frozen-lockfile

# Localize:
- name: Localization build dry run
working-directory: frontend
run: yarn localize:build

build:
name: Build
runs-on: ubuntu-latest
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
strategy:
matrix:
node: [20, 22]
steps:
# Setup:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}

- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: 'yarn'
cache-dependency-path: frontend/yarn.lock

- name: Restore cache
uses: actions/cache@v4
with:
path: frontend/dist
key: ${{ runner.os }}-btrix-frontend-build-${{ hashFiles('frontend/dist') }}
restore-keys: |
${{ runner.os }}-btrix-frontend-build-
- name: Install dependencies
working-directory: frontend
env:
HUSKY: 0
run: yarn install --frozen-lockfile

# Check build:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
with:
driver-opts: network=host

- name: Build Frontend
uses: docker/build-push-action@v3
with:
context: frontend
load: true
tags: webrecorder/browsertrix-frontend:latest
cache-from: type=gha,scope=frontend
cache-to: type=gha,scope=frontend,mode=max
43 changes: 34 additions & 9 deletions .github/workflows/k3d-ci.yaml
Original file line number Diff line number Diff line change
@@ -1,23 +1,48 @@
name: Cluster Run (K3d)

on:
push:
paths:
- 'backend/**'
- 'chart/**'
on: [push, pull_request]

pull_request:
paths:
- 'backend/**'
- 'chart/**'
# Cancel in progress workflows on pull_requests.
# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

env:
ECHO_SERVER_HOST_URL: http://host.k3d.internal:18080

jobs:
# In order to be able to have required checks, a workflow can't be entirely
# skipped: see https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/troubleshooting-required-status-checks#handling-skipped-but-required-checks
paths-filter:
name: "Changed files?"
runs-on: ubuntu-latest
outputs:
matches: ${{ steps.filter.outputs.matches }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2 # important, to fetch previous commit

# workaround for https://github.com/dorny/paths-filter/issues/240
- id: previous-sha
run: 'echo "sha=$(git rev-parse HEAD^1)" >> $GITHUB_OUTPUT'

- uses: dorny/paths-filter@v3
id: filter
with:
base: "${{ steps.previous-sha.outputs.sha }}"
filters: |
matches:
- 'backend/**'
- 'chart/**'
- '.github/workflows/k3d-ci.yaml'
btrix-k3d-test:
timeout-minutes: 45
runs-on: ubuntu-latest
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
steps:
- name: Create k3d Cluster
uses: AbsaOSS/k3d-action@v2
42 changes: 0 additions & 42 deletions .github/workflows/lint.yaml

This file was deleted.

88 changes: 0 additions & 88 deletions .github/workflows/ui-tests-playwright.yml

This file was deleted.

41 changes: 32 additions & 9 deletions .github/workflows/weblate-reformat.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,38 @@
name: Weblate Reformat
on:
pull_request_target:
paths:
- 'frontend/xliff/**'
- '.github/workflows/weblate-reformat.yaml'

branches:
- 'weblate'
pull_request_target

jobs:
setup-and-build:
# In order to be able to have required checks, a workflow can't be entirely
# skipped: see https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/troubleshooting-required-status-checks#handling-skipped-but-required-checks
paths-filter:
name: "Changed files?"
runs-on: ubuntu-latest
# Only run on PRs with a target branch set of 'weblate'
if: github.base_ref == 'weblate'
outputs:
matches: ${{ steps.filter.outputs.matches }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2 # important, to fetch previous commit

# workaround for https://github.com/dorny/paths-filter/issues/240
- id: previous-sha
run: 'echo "sha=$(git rev-parse HEAD^1)" >> $GITHUB_OUTPUT'

- uses: dorny/paths-filter@v3
id: filter
with:
base: "${{ steps.previous-sha.outputs.sha }}"
filters: |
matches:
- 'frontend/xliff/**'
- '.github/workflows/weblate-reformat.yaml'
reformat:
needs: paths-filter
if: needs.paths-filter.outputs.matches == 'true'
runs-on: ubuntu-latest
permissions:
# Give the default GITHUB_TOKEN write permission to commit and push the
@@ -35,7 +58,7 @@ jobs:
env:
HUSKY: 0
run: yarn install --frozen-lockfile

# Localize:
- name: Reformat XLIFF files
working-directory: frontend

0 comments on commit 318acaf

Please sign in to comment.