Skip to content

Commit

Permalink
tests: run the test_stages category in parallel
Browse files Browse the repository at this point in the history
Run the `test_stages` test in parallel in the github runner. This
test currently takes about 1:30h to 2:30h and running it in parallel
will give us big wins in terms of test time. The time is observed
to go down to 0:30h to 1h.

Note that the other tests are not run in parallel. The reason is
that they fail randomly, it looks like insufficient isolation
between them. Some are easy to fix, e.g.:
7215212
but it's probably not worth it as the other tests run a lot faster.
  • Loading branch information
mvo5 committed Nov 7, 2023
1 parent cd1c5f0 commit f6e21e1
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 1 deletion.
11 changes: 11 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,16 @@ jobs:
with:
image: ghcr.io/osbuild/osbuild-ci:latest-202308241910
run: |
# Note that only "test.run.test_stages" runs in parallel because
# the other tests are not sufficiently isolated and will cause
# random failures. But test_stages is the long running one with
# almost 2h.
if [ "${{ matrix.test }}" = "test.run.test_stages" ]; then
# Using 4 workers is a bit arbitrary, "auto" is probably too
# aggressive.
export TEST_WORKERS="-n 4"
# Share the store between the workers speeds things up further
export OSBUILD_TEST_STORE=/var/tmp/osbuild-test-store
fi
TEST_CATEGORY="${{ matrix.test }}" \
tox -e "${{ matrix.environment }}"
3 changes: 2 additions & 1 deletion tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ labels =
description = "run osbuild unit tests"
deps =
pytest
pytest-xdist
jsonschema
mako
iniparse
Expand All @@ -26,7 +27,7 @@ passenv =
TEST_CATEGORY

commands =
bash -c 'python -m pytest --pyargs --rootdir=. {env:TEST_CATEGORY}'
bash -c 'python -m pytest --pyargs --rootdir=. {env:TEST_CATEGORY} {env:TEST_WORKERS}'

allowlist_externals =
bash
Expand Down

0 comments on commit f6e21e1

Please sign in to comment.