tests: run the test_stages category in parallel

Run the `test_stages` test in parallel in the github runner. This
test currently takes about 1:30h to 2:30h and running it in parallel
will give us big wins in terms of test time. The time is observed
to go down to 0:30h to 1h.

Note that the other tests are not run in parallel. The reason is
that they fail randomly, it looks like insufficient isolation
between them. Some are easy to fix, e.g.:
7215212
but it's probably not worth it as the other tests run a lot faster.

.github/workflows/test.yml

@@ -32,5 +32,16 @@ jobs:
       with:
         image: ghcr.io/osbuild/osbuild-ci:latest-202308241910
         run: |
+          # Note that only "test.run.test_stages" runs in parallel because
+          # the other tests are not sufficiently isolated and will cause
+          # random failures. But test_stages is the long running one with
+          # almost 2h.
+          if [ "${{ matrix.test }}" = "test.run.test_stages" ]; then
+              # Using 4 workers is a bit arbitrary, "auto" is probably too
+              # aggressive.
+              export TEST_WORKERS="-n 4"
+              # Share the store between the workers speeds things up further
+              export OSBUILD_TEST_STORE=/var/tmp/osbuild-test-store
+          fi
           TEST_CATEGORY="${{ matrix.test }}" \
           tox -e "${{ matrix.environment }}"

tox.ini

@@ -13,6 +13,7 @@ labels =
 description = "run osbuild unit tests"
 deps =
     pytest
+    pytest-xdist
     jsonschema
     mako
     iniparse
@@ -26,7 +27,7 @@ passenv =
     TEST_CATEGORY
 
 commands =
-    bash -c 'python -m pytest --pyargs --rootdir=. {env:TEST_CATEGORY}'
+    bash -c 'python -m pytest --pyargs --rootdir=. {env:TEST_CATEGORY} {env:TEST_WORKERS}'
 
 allowlist_externals =
     bash