internal/ci: use registry-based GitHub module

This stops using a schema vendored from SchemaStore for GitHub workflows and starts using the experimental module that was published in the Central Registry as part of making the same change for cue-lang/cue, in https://cuelang.org/cl/1201986. The change to the import source was made inside: - internal/ci/base/gerrithub.cue - internal/ci/base/github.cue - internal/ci/gerrithub/gerrithub.cue - internal/ci/github/trybot.cue ... with other changes, described below, made in service of this switch. These updates were required: - json.#step and json.#Workflow became githubactions.#Step and githubactions.#Workflow throughout. - an early step was added to the trybot workflow that causes the CUE module's dependencies to be fetched and cached, through access to NOTCUECKOO_CUE_LOGINS: credentials for an unprivileged user that has read-only access to public modules in the Central Registry. - the cmd/cue version that's invoked to build the site (*not* the version of CUE that the site actually documents) was upgraded via go-get to the latest pre-release (v0.11.0-alpha.4) in order to access the matchN built-in, which is used by the published module. These updates were direct knock-on changes: - the imported vendored schema and its augmentations were deleted. - the file containing only the CUE workflow command that regenerates the vendored schema was deleted, along with the go-generate invocation that used to call the command. These changes weren't strictly necessary, but felt like a good idea: - the imports of the vendored schema originally had an import identifier clash with the "encoding/json" built-in package and the built-in package was chosen to be a named import (encjson instead of json). After this change, that clash no longer occurred, so the built-in package had its named import removed, reverting to simply "encoding/json", and references to it were updated throughout in keeping with the change made in https://cuelang.org/cl/1201986. - "go run cuelang.org/go/cmd/cue fmt ./internal/..." used the CUE pre-release. The CUE pre-release appears to have some harmless YAML export ordering differences that show themselves as a bit of noise in the GitHub workflow files' diff. This change is marked as Preprocessor-No-Write-Cache:true to avoid cache file churn caused by the CUE version upgrade. Preprocessor-No-Write-Cache: true Signed-off-by: Jonathan Matthews <[email protected]> Change-Id: I38286853de0a1f4d25468e4a75a94a3730872c27 Dispatch-Trailer: {"type":"trybot","CL":1202554,"patchset":9,"ref":"refs/changes/54/1202554/9","targetBranch":"master"}
cue-lang · Oct 28, 2024 · 5c5dcc5 · 5c5dcc5
1 parent e625515
commit 5c5dcc5
Show file tree

Hide file tree

Showing 17 changed files with 182 additions and 1,150 deletions.
diff --git a/.github/workflows/evict_caches.yaml b/.github/workflows/evict_caches.yaml
@@ -21,8 +21,8 @@ jobs:
         run: touch -t 202211302355 $(find * -type d)
       - name: Restore git file modification times
         uses: chetan/git-restore-mtime-action@075f9bc9d159805603419d50f794bd9f33252ebe
-      - id: DispatchTrailer
-        name: Try to extract Dispatch-Trailer
+      - name: Try to extract Dispatch-Trailer
+        id: DispatchTrailer
         run: |-
           x="$(git log -1 --pretty='%(trailers:key=Dispatch-Trailer,valueonly)')"
           if [[ "$x" == "" ]]
@@ -38,11 +38,11 @@ jobs:
           echo "value<<EOD" >> $GITHUB_OUTPUT
           echo "$x" >> $GITHUB_OUTPUT
           echo "EOD" >> $GITHUB_OUTPUT
-      - if: |-
+      - name: Check we don't have Dispatch-Trailer on a protected branch
+        if: |-
           ((github.ref == 'refs/heads/master') && (! (contains(github.event.head_commit.message, '
           Dispatch-Trailer: {"type":"')))) && (contains(github.event.head_commit.message, '
           Dispatch-Trailer: {"type":"'))
-        name: Check we don't have Dispatch-Trailer on a protected branch
         run: |-
           echo "github.event.head_commit.message contains Dispatch-Trailer but we are on a protected branch"
           false

diff --git a/.github/workflows/push_tip_to_trybot.yaml b/.github/workflows/push_tip_to_trybot.yaml
@@ -9,10 +9,6 @@ concurrency: push_tip_to_trybot
 jobs:
   push:
     runs-on: ubuntu-22.04
-    defaults:
-      run:
-        shell: bash
-    if: ${{github.repository == 'cue-lang/cuelang.org'}}
     steps:
       - name: Write netrc file for cueckoo Gerrithub
         run: |-
@@ -48,3 +44,7 @@ jobs:
           	 echo "Giving up"
           	 exit 1
           fi
+    defaults:
+      run:
+        shell: bash
+    if: ${{github.repository == 'cue-lang/cuelang.org'}}
diff --git a/.github/workflows/trybot.yaml b/.github/workflows/trybot.yaml
@@ -34,8 +34,8 @@ jobs:
         run: touch -t 202211302355 $(find * -type d)
       - name: Restore git file modification times
         uses: chetan/git-restore-mtime-action@075f9bc9d159805603419d50f794bd9f33252ebe
-      - id: DispatchTrailer
-        name: Try to extract Dispatch-Trailer
+      - name: Try to extract Dispatch-Trailer
+        id: DispatchTrailer
         run: |-
           x="$(git log -1 --pretty='%(trailers:key=Dispatch-Trailer,valueonly)')"
           if [[ "$x" == "" ]]
@@ -51,28 +51,27 @@ jobs:
           echo "value<<EOD" >> $GITHUB_OUTPUT
           echo "$x" >> $GITHUB_OUTPUT
           echo "EOD" >> $GITHUB_OUTPUT
-      - if: |-
+      - name: Check we don't have Dispatch-Trailer on a protected branch
+        if: |-
           ((github.ref == 'refs/heads/master') && (! (contains(github.event.head_commit.message, '
           Dispatch-Trailer: {"type":"')))) && (contains(github.event.head_commit.message, '
           Dispatch-Trailer: {"type":"'))
-        name: Check we don't have Dispatch-Trailer on a protected branch
         run: |-
           echo "github.event.head_commit.message contains Dispatch-Trailer but we are on a protected branch"
           false
-      - if: github.event.inputs.scheduled == 'true'
-        name: Fail if Preprocessor-No-Write-Cache trailer is present for a scheduled workflow run
+      - name: Fail if Preprocessor-No-Write-Cache trailer is present for a scheduled workflow run
+        if: github.event.inputs.scheduled == 'true'
         run: '! ./_scripts/noWriteCache.bash HEAD'
       - if: github.repository != 'cue-lang/cuelang.org'
         run: |
           echo 'Setting CI_NO_SKIP_CACHE=true'
           echo "CI_NO_SKIP_CACHE=true" >> $GITHUB_ENV
       - if: runner.os == 'macOS'
-        name: Set TMPDIR environment variable (${{runner.os}})
         run: |-
           mkdir $HOME/.tmp
           echo "TMPDIR=$HOME/.tmp" >> $GITHUB_ENV
+        name: Set TMPDIR environment variable (${{runner.os}})
       - if: runner.os == 'macOS'
-        name: Write lima config (${{runner.os}})
         run: |-
           mkdir -p ~/.lima/default
           cat <<EOD > ~/.lima/default/lima.yaml
@@ -82,15 +81,16 @@ jobs:
             - location: "$TMPDIR"
           	 writable: true
           EOD
+        name: Write lima config (${{runner.os}})
       - if: runner.os == 'macOS'
-        name: Install Docker (${{runner.os}})
         run: |-
           brew install colima docker
           colima start --mount-type virtiofs
           sudo ln -sf $HOME/.colima/default/docker.sock /var/run/docker.sock
+        name: Install Docker (${{runner.os}})
       - if: runner.os == 'macOS'
-        name: Set DOCKER_HOST environment variable (${{runner.os}})
         run: echo "DOCKER_HOST=unix://$HOME/.colima/default/docker.sock" >> $GITHUB_ENV
+        name: Set DOCKER_HOST environment variable (${{runner.os}})
       - if: runner.os == 'macOS'
         name: Install macOS utils
         run: brew install coreutils
@@ -127,17 +127,13 @@ jobs:
           	echo 'Found Preprocessor-No-Write-Cache trailer'
           	echo "PREPROCESSOR_NOWRITECACHE=true" >> $GITHUB_ENV
           fi
-      - id: go-mod-cache-dir
-        name: Get go mod cache directory
+      - name: Get go mod cache directory
+        id: go-mod-cache-dir
         run: echo "dir=$(go env GOMODCACHE)" >> ${GITHUB_OUTPUT}
-      - id: go-cache-dir
-        name: Get go build/test cache directory
+      - name: Get go build/test cache directory
+        id: go-cache-dir
         run: echo "dir=$(go env GOCACHE)" >> ${GITHUB_OUTPUT}
-      - if: |-
-          (((github.ref == 'refs/heads/master') && (! (contains(github.event.head_commit.message, '
-          Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test'))
-        uses: actions/cache@v4
-        with:
+      - with:
           path: |-
             ${{ steps.go-mod-cache-dir.outputs.dir }}/cache/download
             ${{ steps.go-cache-dir.outputs.dir }}
@@ -147,11 +143,11 @@ jobs:
             ${{ github.workspace }}/playground/.webpack_cache
           key: ${{ runner.os }}-1.23.0-${{ github.run_id }}
           restore-keys: ${{ runner.os }}-1.23.0
-      - if: |-
-          ! (((github.ref == 'refs/heads/master') && (! (contains(github.event.head_commit.message, '
+        if: |-
+          (((github.ref == 'refs/heads/master') && (! (contains(github.event.head_commit.message, '
           Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test'))
-        uses: actions/cache/restore@v4
-        with:
+        uses: actions/cache@v4
+      - with:
           path: |-
             ${{ steps.go-mod-cache-dir.outputs.dir }}/cache/download
             ${{ steps.go-cache-dir.outputs.dir }}
@@ -161,6 +157,10 @@ jobs:
             ${{ github.workspace }}/playground/.webpack_cache
           key: ${{ runner.os }}-1.23.0-${{ github.run_id }}
           restore-keys: ${{ runner.os }}-1.23.0
+        uses: actions/cache/restore@v4
+        if: |-
+          ! (((github.ref == 'refs/heads/master') && (! (contains(github.event.head_commit.message, '
+          Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test'))
       - if: |-
           github.repository == 'cue-lang/cuelang.org' && (((github.ref == 'refs/heads/master') && (! (contains(github.event.head_commit.message, '
           Dispatch-Trailer: {"type":"')))) || github.ref == 'refs/heads/ci/test')
@@ -169,15 +169,23 @@ jobs:
         run: go run cuelang.org/go/internal/ci/[email protected]
       - name: Perform early content checks
         run: _scripts/checkContent.bash
+      - name: Populate CUE dependency cache
+        env:
+          CUE_LOGINS: ${{ secrets.NOTCUECKOO_CUE_LOGINS }}
+        run: |-
+          export CUE_CONFIG_DIR="$(mktemp -d)"
+          echo "$CUE_LOGINS" >"$CUE_CONFIG_DIR/logins.json"
+          go run cuelang.org/go/cmd/cue mod tidy --check
+          rm -rfv "${CUE_CONFIG_DIR}"
       - name: Check site CUE configuration
         run: _scripts/runPreprocessor.bash execute --check
       - name: Regenerate
         run: go generate ./...
       - name: Regenerate Playground
         run: go generate ./...
         working-directory: playground
-      - if: always()
-        name: Check that git is clean at the end of the job
+      - name: Check that git is clean at the end of the job
+        if: always()
         run: test -z "$(git status --porcelain)" || (git status; git diff; false)
       - run: ./_scripts/buildDockerImage.bash
       - run: npm install
@@ -205,16 +213,16 @@ jobs:
           EOD
       - name: Dist
         run: ./_scripts/build.bash --baseURL https://cl-${{ fromJSON(steps.DispatchTrailer.outputs.value).CL }}-${{ fromJSON(steps.DispatchTrailer.outputs.value).patchset }}--cue-cls.netlify.app
-      - if: always()
-        name: Check that git is clean at the end of the job
+      - name: Check that git is clean at the end of the job
+        if: always()
         run: test -z "$(git status --porcelain)" || (git status; git diff; false)
       - run: ./playground/_scripts/checkCUEVersion.bash
-      - run: npm run lint
-        working-directory: hugo
-      - if: |-
+      - working-directory: hugo
+        run: npm run lint
+      - name: Install Netlify CLI
+        if: |-
           github.repository == 'cue-lang/cuelang.org-trybot' && (contains(github.event.head_commit.message, '
           Dispatch-Trailer: {"type":"trybot"'))
-        name: Install Netlify CLI
         run: npm install -g [email protected]
       - if: |-
           github.repository == 'cue-lang/cuelang.org-trybot' && (contains(github.event.head_commit.message, '
@@ -233,18 +241,18 @@ jobs:
           ALGOLIA_INDEX_FILE: ../_public/algolia.json
       - name: 'tip.cuelang.org: Patch the site to be compatible with the tip of cue-lang/cue'
         run: _scripts/tipPatchApply.bash
-      - if: github.repository == 'cue-lang/cuelang.org' && (github.ref == 'refs/heads/master' || (github.ref == 'refs/heads/ci/test'))
-        name: 'tip.cuelang.org: Configure the site to use the tip of cue-lang/cue'
-        run: _scripts/tipUseAlternativeCUE.bash
+      - name: 'tip.cuelang.org: Configure the site to use the tip of cue-lang/cue'
+        if: github.repository == 'cue-lang/cuelang.org' && (github.ref == 'refs/heads/master' || (github.ref == 'refs/heads/ci/test'))
         env:
           GOPRIVATE: cuelang.org/go
-      - if: github.repository == 'cue-lang/cuelang.org' && (github.ref == 'refs/heads/master' || (github.ref == 'refs/heads/ci/test'))
-        name: 'tip.cuelang.org: Build the site against the tip of cue-lang/cue'
+        run: _scripts/tipUseAlternativeCUE.bash
+      - name: 'tip.cuelang.org: Build the site against the tip of cue-lang/cue'
+        if: github.repository == 'cue-lang/cuelang.org' && (github.ref == 'refs/heads/master' || (github.ref == 'refs/heads/ci/test'))
         run: _scripts/regenPostInfraChange.bash
         env:
           GOPRIVATE: cuelang.org/go
-      - if: github.repository == 'cue-lang/cuelang.org' && (github.ref == 'refs/heads/master' || (github.ref == 'refs/heads/ci/test'))
-        name: 'tip.cuelang.org: Deploy the site'
+      - name: 'tip.cuelang.org: Deploy the site'
+        if: github.repository == 'cue-lang/cuelang.org' && (github.ref == 'refs/heads/master' || (github.ref == 'refs/heads/ci/test'))
         run: |-
           git config http.https://github.com/.extraheader "AUTHORIZATION: basic $(echo -n cueckoo:${{ secrets.CUECKOO_GITHUB_PAT }} | base64)"
           _scripts/tipDeploy.bash 'cueckoo' '[email protected]'

diff --git a/.github/workflows/trybot_dispatch.yaml b/.github/workflows/trybot_dispatch.yaml
@@ -9,10 +9,10 @@ name: Dispatch trybot
 jobs:
   trybot:
     runs-on: ubuntu-22.04
+    if: ${{ ((github.ref == 'refs/heads/ci/test') && false) || github.event.client_payload.type == 'trybot' }}
     defaults:
       run:
         shell: bash
-    if: ${{ ((github.ref == 'refs/heads/ci/test') && false) || github.event.client_payload.type == 'trybot' }}
     steps:
       - name: Write netrc file for cueckoo Gerrithub
         run: |-
@@ -22,17 +22,17 @@ jobs:
           password ${{ secrets.CUECKOO_GERRITHUB_PASSWORD }}
           EOD
           chmod 600 ~/.netrc
-      - id: payload
+      - name: Write fake payload
+        id: payload
         if: github.repository == 'cue-lang/cuelang.org' && (github.ref == 'refs/heads/ci/test')
-        name: Write fake payload
         run: |-
           cat <<EOD >> $GITHUB_OUTPUT
           value<<DOE
           null
           DOE
           EOD
-      - if: github.event.client_payload.type != 'trybot'
-        name: Trigger TryBot (fake data)
+      - name: Trigger TryBot (fake data)
+        if: github.event.client_payload.type != 'trybot'
         run: |-
           mkdir tmpgit
           cd tmpgit
@@ -81,8 +81,8 @@ jobs:
           	echo "Giving up"
           	exit 1
           fi
-      - if: github.event.client_payload.type == 'trybot'
-        name: Trigger TryBot (repository_dispatch payload)
+      - name: Trigger TryBot (repository_dispatch payload)
+        if: github.event.client_payload.type == 'trybot'
         run: |-
           mkdir tmpgit
           cd tmpgit

diff --git a/cue.mod/module.cue b/cue.mod/module.cue
@@ -2,3 +2,8 @@ module: "github.com/cue-lang/cuelang.org@v0"
 language: {
 	version: "v0.9.0"
 }
+deps: {
+	"github.com/cue-tmp/jsonschema-pub/exp1/githubactions@v0": {
+		v: "v0.3.0"
+	}
+}