Fix coverage link, streamline env vars in CI #12971
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # These variables are overridden in CI with GitHub Action variables if triggered from pull-request or push; | ||
| # Locally, fallback to the current local git status | ||
| BRANCH="$(git branch --show-current)" | ||
| COMMIT_SHA="$(git rev-parse HEAD)" | ||
|
|
||
| # Read the CesiumJS version from package.json | ||
| CESIUM_VERSION=$(npm pkg get version | tr -d '"') | ||
|
|
||
| # Build artifact configuration | ||
| BUILD_ARTIFACT_PATH="cesium/${BRANCH}" | ||
| BUILD_ARTIFACT_BUCKET="s3://cesium-public-builds/${BUILD_ARTIFACT_PATH}" | ||
| BUILD_ARTIFACT_URL="https://ci-builds.cesium.com/${BUILD_ARTIFACT_PATH}" | ||
|
|
||
| INDEX_URL="${BUILD_ARTIFACT_URL}/index.html" | ||
| ZIP_URL="${BUILD_ARTIFACT_URL}/Cesium-${CESIUM_VERSION}.zip" | ||
| NPM_URL="${BUILD_ARTIFACT_URL}/cesium-${CESIUM_VERSION}.tgz" | ||
| COVERAGE_URL="${BUILD_ARTIFACT_URL}/Build/Coverage/index.html" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| BRANCH="$(jq -r .pull_request.head.ref ${GITHUB_EVENT_PATH})" | ||
| COMMIT_SHA="$(jq -r .pull_request.head.sha ${GITHUB_EVENT_PATH})" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| BRANCH=${GITHUB_REF_NAME} | ||
| COMMIT_SHA=${GITHUB_SHA} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: coverage | ||
| on: | ||
| pull_request: | ||
| concurrency: | ||
| group: coverage-${{ github.ref }} | ||
| cancel-in-progress: true | ||
| jobs: | ||
| coverage: | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| AWS_ACCESS_KEY_ID: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }} | ||
| AWS_SECRET_ACCESS_KEY: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }} | ||
| AWS_REGION: us-east-1 | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| permissions: | ||
| statuses: write | ||
| contents: read | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
| - name: install node 22 | ||
| uses: actions/setup-node@v5 | ||
| with: | ||
| node-version: "22" | ||
| - name: npm install | ||
| run: npm install | ||
| - name: set status pending | ||
| if: ${{ env.AWS_ACCESS_KEY_ID != '' }} | ||
| run: | | ||
| npx @dotenvx/dotenvx run -f ./.github/workflows/.env.pull_request -- \ | ||
| node ./scripts/setCommitStatus.js coverage pending | ||
| - name: build | ||
| run: npm run build | ||
| - name: coverage (firefox) | ||
| run: npm run coverage -- --browsers FirefoxHeadless --webgl-stub --failTaskOnError --suppressPassed | ||
| - name: upload coverage artifacts | ||
| if: ${{ env.AWS_ACCESS_KEY_ID != '' }} | ||
| run: | | ||
| npx @dotenvx/dotenvx run -f ./.github/workflows/.env.pull_request -f .env \ | ||
| -- sh -c \ | ||
| 'aws s3 sync ./Build/Coverage $BUILD_ARTIFACT_BUCKET/Build/Coverage --delete' | ||
| - name: set status success | ||
| if: ${{ env.AWS_ACCESS_KEY_ID != '' }} | ||
| run: | | ||
| npx @dotenvx/dotenvx run -f ./.github/workflows/.env.pull_request -- \ | ||
| node ./scripts/setCommitStatus.js coverage ${{ job.status }} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,7 +15,7 @@ jobs: | |
| - name: install node 22 | ||
| uses: actions/setup-node@v5 | ||
| with: | ||
| node-version: "22" | ||
| - name: npm install | ||
| run: npm install | ||
| - name: lint *.js | ||
|
|
@@ -24,36 +24,14 @@ jobs: | |
| run: npm run markdownlint | ||
| - name: format code | ||
| run: npm run prettier-check | ||
| release-tests: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
| - name: install node 22 | ||
| uses: actions/setup-node@v5 | ||
| with: | ||
| node-version: "22" | ||
| - name: npm install | ||
| run: npm install | ||
| - name: release build | ||
|
|
@@ -69,7 +47,7 @@ jobs: | |
| - name: install node 20 | ||
| uses: actions/setup-node@v5 | ||
| with: | ||
| node-version: "20" | ||
| - name: npm install | ||
| run: npm install | ||
| - name: release build | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -29,7 +29,6 @@ jobs: | |
| AWS_REGION: us-east-1 | ||
| BRANCH: ${{ github.ref_name }} | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| GITHUB_SHA: ${{ github.sha }} | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,7 +13,6 @@ jobs: | |
| AWS_REGION: us-east-1 | ||
| BRANCH: ${{ github.ref_name }} | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| GITHUB_SHA: ${{ github.sha }} | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -153,6 +153,3 @@ Here's the full set of scripts and what they do. | |
| - `test-webgl-stub` - Runs all tests using the WebGL stub, which WebGL calls a noop and ignores related test expectations | ||
| - `test-webgl-validation` - Runs all tests with Karma and enables low-level WebGL validation | ||
| - `test-release` - Runs all tests on the minified release version of built Cesium | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,8 +2,9 @@ | |
|
|
||
| - [Background](#background) | ||
| - [Actions and workflows](#actions-and-workflows) | ||
| - [Environment variables](#environment-variables) | ||
| - [Continuous deployment](#continuous-deployment) | ||
| - [Configuration guide](#configuration-guide) | ||
| - [Configure a different S3 bucket](#configure-a-different-s3-bucket) | ||
| - [Configure S3 credentials](#configure-s3-credentials) | ||
|
|
||
|
|
@@ -18,51 +19,52 @@ CesiumJS uses [GitHub Actions](https://docs.github.com/en/actions) for continuou | |
|
|
||
| Reusable actions are defined in `/.github/actions/` and workflows in `.github/workflows/`. | ||
|
|
||
| Workflows are triggered when a commit is pushed to the `cesium` repository and when a contributor opens or updates a pull request. After the build has completed, the overall status of the build is shown at the bottom of the pull request page. | ||
|
|
||
| In the dropdown menu, individual status checks are displayed. Logs and deployed build artifacts can be accessed by clicking the link associated with the individual check. | ||
|
|
||
|  | ||
|
|
||
| The status checks for any branch are also accessible under the [Branches](https://github.com/CesiumGS/cesium/branches/all) page by clicking the icon next to the branch name. | ||
|
|
||
|  | ||
|
|
||
| ### Environment variables | ||
|
|
||
| Any non-secret environment variables for CI and CD are managed using `.env` files and [dotenvx](https://github.com/dotenvx/dotenvx). | ||
|
|
||
| - **Local dev**: `.env` is checked in at the repository root and provides defaults constant values and local fallbacks. | ||
| - To run a script or command with these variables configured locally, use `npx @dotenvx/dotenvx run -- <COMMAND>`, e.g., `npx @dotenvx/dotenvx run -- node ./scripts/setCommitStatus.js`. | ||
| - To expand a variable directly in a command, use a subshell command, e.g., `npx @dotenvx/dotenvx run -- sh -c 'echo "${BRANCH}"'`. | ||
| - **GitHub Actions workflow - `push` trigger**: `.github/workflows/.env.push` | ||
| - **GitHub Actions workflow - `pull_request` trigger**: `.github/workflows/.env.pull_request` | ||
|
|
||
| ## Continuous deployment | ||
|
|
||
| Automated deployments make recent code changes available for convenient testing and review—No need to fetch or build locally. In the `cesium` repository, all continuous deployment artifacts are uploaded for commits authored by users with commit access. | ||
|
|
||
| Each of the following are deployed on a per-branch basis. | ||
|
|
||
| | Artifact | Link pattern (`main`) | | ||
| | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ||
| | Sandcastle | `https://ci-builds.cesium.com/cesium/<BRANCH>/Apps/Sandcastle/index.html` (i.e., [`https://ci-builds.cesium.com/cesium/main/Apps/Sandcastle/index.html`](https://ci-builds.cesium.com/cesium/main/Apps/Sandcastle/index.html)) | | ||
| | Documentation | `https://ci-builds.cesium.com/cesium/<BRANCH>/Build/Documentation/index.html` (i.e., [`https://ci-builds.cesium.com/cesium/main/Build/Documentation/index.html`](https://ci-builds.cesium.com/cesium/main/Build/Documentation/index.html)) | | ||
| | Coverage results | `https://ci-builds.cesium.com/cesium/<BRANCH>/Build/Coverage/index.html` (i.e., [`https://ci-builds.cesium.com/cesium/main/Build/Coverage/index.html`](https://ci-builds.cesium.com/cesium/main/Build/Coverage/index.html)) | | ||
| | Release zip | `https://ci-builds.cesium.com/cesium/<BRANCH>/Cesium-<VERSION>-<BRANCH>.0.zip` (i.e., [`https://ci-builds.cesium.com/cesium/main/Cesium-1.X.X-main.0.zip`](https://ci-builds.cesium.com/cesium/main/Cesium-1.X.X-main.0.zip)) | | ||
| | npm package | `https://ci-builds.cesium.com/cesium/<BRANCH>/cesium-<VERSION>-<BRANCH>.0.tgz` (i.e., [`https://ci-builds.cesium.com/cesium/main/cesium-1.X.X-main.0.tgz`](https://ci-builds.cesium.com/cesium/main/cesium-1.X.X-main.0.tgz)) | | ||
|
|
||
| ## Configuration guide | ||
|
|
||
| Additional set up is required for deployment _only_ if you do not have commit access to CesiumJS. | ||
|
|
||
| ### Configure a different S3 bucket | ||
|
|
||
| It is possible to configure your development branch of CesiumJS to deploy build artifacts to a different [AWS S3 Bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html). If you are using the default "cesium-public-builds" bucket and have valid credentials, skip to [Configure S3 Credentials](#configure-s3-credentials) | ||
|
|
||
| In the environment file `.env`, update the following values: | ||
|
|
||
| - Replace the value of `BUILD_ARTIFACT_BUCKET` with the name of the target S3 bucket. | ||
| - Replace the value of `BUILD_ARTIFACT_URL` with the public URL correspondeding to the target S3 bucket. | ||
|
|
||
| ### Configure S3 credentials | ||
|
|
||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we calling
dotenvxfromnpxinstead of using it directly in the script files? is it just so we can specify the extra.env.pull_requestfiles?Also as an aside for my curiosity, what is the reason you went with
dotenvxinstead of justdotenv? a quick search seems that the motivation would be to encrypt the env files with secrets but we aren't doing that. Is this future proofing?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I chose
dotenvxspecifically because it supports variable expansion and command substitution, which I don't believe "regular"dotenvsupports.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. The default
.envis being loaded from the script itself, but this command is what allows us to override with the CI-specific values in.env.pull_request. If there's an alternative syntax you know of, please tell!