Merge #6400: ci: refactor GitHub Actions CI to use reusable workflows…

…, respect `DEP_OPTS`, build multiprocess depends with Clang

a976777 ci: build multiprocess depends with Clang, use in tsan builds (Kittywhiskers Van Gogh)
c9708b7 ci: drop `DEBUG=1` to try and avoid exhausting Actions runner storage (Kittywhiskers Van Gogh)
68ddcdb ci: add nowallet depends, multiprocess depends and build variant (Kittywhiskers Van Gogh)
c83efa8 ci: start respecting `DEP_OPTS`, unify with GitLab (Kittywhiskers Van Gogh)
f41efa4 ci: source target triple from `matrix.sh` (Kittywhiskers Van Gogh)
a7d11f8 ci: pass the whole key instead of trying to reconstruct it (Kittywhiskers Van Gogh)
fa0dc5e ci: switch from matrices to manual definition (Kittywhiskers Van Gogh)
1f58365 ci: split out building src and running unit tests into reusable workflow (Kittywhiskers Van Gogh)
0d8f2ef ci: split out building depends into reusable workflow (Kittywhiskers Van Gogh)
41d9867 ci: split out building CI container into reusable workflow (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Depends on #6406

  * We have to move away from matrices as matrix-based job variants cannot be _individually_ accepted as dependencies for the next job, only the whole job can. This means that the next job cannot start until the last matrix variant of the dependency succeeds. This also means that if any variant fails in the dependency job fails, every job forward also ceases.

    The approach taken in this PR is a mirror of the current implementation on GitLab CI, defining a reusable unit ("templates" in GitLab parlance, "reusable workflows" in GitHub) and defining each variant, now coming with a unique job name that can be specified as dependencies for the job. This should reduce stalling from slower variants.

  * `DEBUG=1` had to be dropped as it resulted in the runners running out of free space ([build](https://github.com/kwvg/dash/actions/runs/13240781702/job/36956549082#step:7:3137)). Despite this only being a problem with GitHub Actions, this change also affects GitLab CI as we now read `DEP_OPTS` from matrix scripts instead of the CI configuration file.
    * We are reading from the matrix script to ensure that both CI instances build with the same configuration and to reduce code complexity arising from passing around details between different workflows.

    * Potential `DEP_OPTS` conflicts (as discussed [here](#6400 (comment))) have been circumvented by passing the expected cache key wholesale instead of reconstructing it ([commit](1a67a02)). We do still expect that `HOST` will be defined correctly in the matrix script (the `HOST` of both depends and build should match) but as differing `HOST`s are incompatible regardless, this shouldn't be a problem.

      * To make sure we can `output` `cache-primary-key`, the depends cache step was split in two so that the output from the `restore` action is can be exported from the workflow  ([source](https://github.com/dashpay/dash/blob/1a67a026de39017a544cfb7852390de88864deec/.github/workflows/build-depends.yml#L21)).

  * As `DEP_OPTS` is a new parameter that _could_ require rebuilding depends, it needs to be incorporated into the cache key. The most straightforward way to do it is to append the hash of the file that defines `DEP_OPTS` to the cache key. Unfortunately, this requires us to hardcode the name of the individual file (e.g., `00_setup_env_native_qt5.sh` for `linux64`) as `matrix.sh` is just a dispatcher and comes with the drawback that _any_ change to the script could result in a cache miss due to a changed hash.

    This has been mitigated by hashing the build variant name and the environment variables that influence the build ([source](https://github.com/dashpay/dash/blob/cfdd2c678819de04cd867740a0f9a9f729c891bb/.github/workflows/build-depends.yml#L41-L43)).
    * `head` trims the output of `sha256sum` to 64 characters, the hash itself isn't being trimmed but everything after it is (trailing hyphen and newline). This is also why `echo -n` is used in some places, to avoid newline addition resulting in a different hash value.

  * Currently there doesn't seem to be a way to have full control on a reusable workflow's name, it remains a fixed "[caller name] / [reusable workflow name]" and attempting to remove the name altogether will still result in a trailing slash ([source](https://stackoverflow.com/questions/79241079/how-to-control-the-name-of-a-reusable-workflow)).

  ## Breaking Changes

  None expected.

  ## Checklist

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    ACK a976777
  PastaPastaPasta:
    utACK a976777

Tree-SHA512: 34436d7e887bc12b89cced49c17ec83405c074dd01eddce8a425c4edc67e7fc6005d1ddc36eef0c8c4888d65a5e6c99ae4bce1726ebfa07d8b7a07063fed4f40

.github/workflows/build-container.yml

@@ -0,0 +1,54 @@
+name: Build container
+
+on:
+  workflow_call:
+    outputs:
+      path:
+        description: "Path to built container"
+        value: ghcr.io/${{ jobs.build.outputs.repo }}/dashcore-ci-runner:${{ jobs.build.outputs.tag }}
+
+env:
+  DOCKER_DRIVER: overlay2
+
+jobs:
+  build:
+    name: Build container
+    runs-on: ubuntu-22.04
+    outputs:
+      tag: ${{ steps.prepare.outputs.tag }}
+      repo: ${{ steps.prepare.outputs.repo }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+
+      - name: Prepare variables
+        id: prepare
+        run: |
+          BRANCH_NAME=$(echo "${GITHUB_REF##*/}" | tr '[:upper:]' '[:lower:]')
+          REPO_NAME=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]')
+          echo "tag=${BRANCH_NAME}" >> $GITHUB_OUTPUT
+          echo "repo=${REPO_NAME}" >> $GITHUB_OUTPUT
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v6
+        with:
+          context: ./contrib/containers/ci
+          file: ./contrib/containers/ci/Dockerfile
+          push: true
+          tags: |
+            ghcr.io/${{ steps.prepare.outputs.repo }}/dashcore-ci-runner:${{ steps.prepare.outputs.tag }}
+            ghcr.io/${{ steps.prepare.outputs.repo }}/dashcore-ci-runner:latest
+          cache-from: type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo }}/dashcore-ci-runner:latest
+          cache-to: type=inline

.github/workflows/build-depends.yml

@@ -0,0 +1,79 @@
+name: Build depends
+
+on:
+  workflow_call:
+    inputs:
+      build-target:
+        description: "Target name as defined by matrix.sh"
+        required: true
+        type: string
+      container-path:
+        description: "Path to built container at registry"
+        required: true
+        type: string
+    outputs:
+      key:
+        description: "Key needed for restoring depends cache"
+        value: ${{ jobs.build-depends.outputs.key }}
+
+jobs:
+  build-depends:
+    name: Build depends
+    runs-on: ubuntu-22.04
+    outputs:
+      key:  ${{ steps.restore.outputs.cache-primary-key }}
+    container:
+      image: ${{ inputs.container-path }}
+      options: --user root
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+
+      - name: Initial setup
+        id: setup
+        run: |
+          BUILD_TARGET="${{ inputs.build-target }}"
+          source ./ci/dash/matrix.sh
+          echo "DEP_OPTS=${DEP_OPTS}" >> "${GITHUB_OUTPUT}"
+          echo "HOST=${HOST}" >> "${GITHUB_OUTPUT}"
+          DEP_HASH="$(echo -n "${BUILD_TARGET}" "${DEP_OPTS}" "${HOST}" | sha256sum | head -c 64)"
+          echo "\"${BUILD_TARGET}\" has HOST=\"${HOST}\" and DEP_OPTS=\"${DEP_OPTS}\" with hash \"${DEP_HASH}\""
+          echo "DEP_HASH=${DEP_HASH}" >> "${GITHUB_OUTPUT}"
+
+        shell: bash
+
+      - name: Cache depends sources
+        uses: actions/cache@v4
+        with:
+          path: |
+            depends/sources
+          key: depends-sources-${{ hashFiles('depends/packages/*') }}
+          restore-keys: |
+            depends-sources-${{ hashFiles('depends/packages/*') }}
+            depends-sources-
+
+      - name: Restore cached depends
+        uses: actions/cache/restore@v4
+        id: restore
+        with:
+          path: |
+            depends/built
+            depends/${{ steps.setup.outputs.HOST }}
+          key: ${{ runner.os }}-depends-${{ inputs.build-target }}-${{ steps.setup.outputs.DEP_HASH }}-${{ hashFiles('depends/packages/*') }}
+          restore-keys: |
+            ${{ runner.os }}-depends-${{ inputs.build-target }}-${{ hashFiles('depends/packages/*') }}
+            ${{ runner.os }}-depends-${{ inputs.build-target }}
+
+      - name: Build depends
+        run: env ${{ steps.setup.outputs.DEP_OPTS }} HOST=${{ steps.setup.outputs.HOST }} make -j$(nproc) -C depends
+
+      - name: Save depends cache
+        uses: actions/cache/save@v4
+        if: steps.restore.outputs.cache-hit != 'true'
+        with:
+          path: |
+            depends/built
+            depends/${{ steps.setup.outputs.HOST }}
+          key: ${{ steps.restore.outputs.cache-primary-key }}

.github/workflows/build-src.yml

@@ -0,0 +1,84 @@
+name: Build source
+
+on:
+  workflow_call:
+    inputs:
+      build-target:
+        description: "Target name as defined by inputs.sh"
+        required: true
+        type: string
+      container-path:
+        description: "Path to built container at registry"
+        required: true
+        type: string
+      depends-key:
+        description: "Key needed to access cached depends"
+        required: true
+        type: string
+
+jobs:
+  build-src:
+    name: Build source
+    runs-on: ubuntu-22.04
+    container:
+      image: ${{ inputs.container-path }}
+      options: --user root
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          fetch-depth: 0
+
+      - name: Initial setup
+        id: setup
+        run: |
+          git config --global --add advice.detachedHead false
+          git config --global --add safe.directory "$PWD"
+          GIT_HEAD="$(git rev-parse HEAD)"
+          git checkout develop
+          git checkout "${GIT_HEAD}"
+          BUILD_TARGET="${{ inputs.build-target }}"
+          source ./ci/dash/matrix.sh
+          echo "HOST=${HOST}" >> $GITHUB_OUTPUT
+          echo "PR_BASE_SHA=${{ github.event.pull_request.base.sha || '' }}" >> $GITHUB_OUTPUT
+        shell: bash
+
+      - name: Restore depends cache
+        uses: actions/cache/restore@v4
+        with:
+          path: |
+            depends/built
+            depends/${{ steps.setup.outputs.HOST }}
+          key: ${{ inputs.depends-key }}
+          fail-on-cache-miss: true
+
+      - name: Manage ccache
+        uses: actions/cache@v4
+        with:
+          path: |
+            /cache
+          key: ${{ runner.os }}-${{ inputs.build-target }}-${{ github.sha }}
+          restore-keys: |
+            ${{ runner.os }}-${{ inputs.build-target }}-${{ github.sha }}
+            ${{ runner.os }}-${{ inputs.build-target }}-${{ steps.setup.outputs.HOST }}
+            ${{ runner.os }}-${{ inputs.build-target }}
+
+      - name: Build source and run unit tests
+        run: |
+          CCACHE_SIZE="400M"
+          CACHE_DIR="/cache"
+          mkdir /output
+          BASE_OUTDIR="/output"
+          BUILD_TARGET="${{ inputs.build-target }}"
+          source ./ci/dash/matrix.sh
+          ./ci/dash/build_src.sh
+          ./ci/dash/test_unittests.sh
+        shell: bash
+
+      - name: Upload build artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: build-artifacts-${{ inputs.build-target }}
+          path: |
+            /output