Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(github): migrate ci from circleci to github #1852

Merged
merged 26 commits into from
Jun 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
a334315
ci(github): migrate ci from circleci to github
rfprod Jun 8, 2022
9cee7e1
ci(cleanup): remove .npmrc after publishing packages
rfprod Jun 8, 2022
5858aeb
ci(github): revise github actions ci workflows
rfprod Jun 8, 2022
4b50606
Update .github/workflows/get-variables.yml
rfprod Jun 10, 2022
5fb7e8d
Update .github/workflows/pr-validation.yml
rfprod Jun 10, 2022
5058adf
ci(premerge): upload premerge unit test coverage results to code climate
rfprod Jun 10, 2022
4ffe17f
ci(dry): setup jobs with a reusable workflow, use matrix for scripts
rfprod Jun 10, 2022
0a08b13
ci(logging): echo explanation of the 'find ... shred ...' command
rfprod Jun 11, 2022
edff7ff
ci(artifacts): check build artifacts workflow
rfprod Jun 11, 2022
9130bb8
ci(setup): fix setup workflow - get node version from inputs
rfprod Jun 11, 2022
f63ad6c
ci(artifacts): save app/integrations dists to workspace cache
rfprod Jun 11, 2022
1d6db8b
ci(actions): convert reusable workflows to composite actions
rfprod Jun 11, 2022
2de8269
ci(actions): fix composite actions
rfprod Jun 11, 2022
4f6c585
ci(actions): update comments in action configs
rfprod Jun 11, 2022
600c297
ci(docs): fix comments in composite actions
rfprod Jun 11, 2022
2dd5a47
ci(syntax): fix yaml syntax errors
rfprod Jun 12, 2022
b8c3086
ci(checkout): checkout sources before calling the setup action
rfprod Jun 12, 2022
d242816
ci(inputs): fix the setup action inputs
rfprod Jun 12, 2022
c0d9f05
ci(syntax): fix the setup action syntax
rfprod Jun 12, 2022
7603b47
ci(test): test:types needs *-build, move it to integration-test
rfprod Jun 12, 2022
4bee88b
ci(cache): fix workspace cache paths
rfprod Jun 12, 2022
95c3034
ci(cache): exclude node_modules in the integrations dir from cache
rfprod Jun 12, 2022
8b53500
ci(cache): fix workspace cache, remove unneeded code
rfprod Jun 12, 2022
a75f815
ci(cache): fix workspace cache
rfprod Jun 12, 2022
b3ba642
ci(actions): add composite actions to upload/download artifacts
rfprod Jun 13, 2022
5eb6e7c
fix: bundlesize step order (release & trunk)
markwhitfeld Jun 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/actions/check-build-artifacts/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# For more information see:
# - https://docs.github.com/en/actions/creating-actions/creating-a-composite-action
# - https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#outputsoutput_id

# The action can be referenced in workflows like:
# - ngxs/store/.github/actions/check-build-artifacts@master
# - ./.github/actions/check-build-artifacts

name: check-build-artifacts
description: Ensure the build artifact directories exist

runs:
using: 'composite'
steps:
- name: Check @ngxs build artifacts
shell: bash
run: |
echo "Making sure the build artifact directories exist."
[[ -d $DEVTOOLS_PLUG_ART ]] && echo "$DEVTOOLS_PLUG_ART $OK" || echo "$DEVTOOLS_PLUG_ART $FAILURE" exit 1
[[ -d $FORM_PLUG_ART ]] && echo "$FORM_PLUG_ART $OK" || echo "$DEVTOOLS_PLUG_ART $FAILURE" exit 1
[[ -d $HMR_PLUG_ART ]] && echo "$HMR_PLUG_ART $OK" || echo "$HMR_PLUG_ART $FAILURE" exit 1
[[ -d $LOGGER_PLUG_ART ]] && echo "$LOGGER_PLUG_ART $OK" || echo "$LOGGER_PLUG_ART $FAILURE" exit 1
[[ -d $ROUTER_PLUG_ART ]] && echo "$ROUTER_PLUG_ART $OK" || echo "$ROUTER_PLUG_ART $FAILURE" exit 1
[[ -d $STORAGE_PLUG_ART ]] && echo "$STORAGE_PLUG_ART $OK" || echo "$STORAGE_PLUG_ART $FAILURE" exit 1
[[ -d $STORE_ART ]] && echo "$STORE_ART $OK" || echo "$STORE_ART $FAILURE" exit 1
[[ -d $WEBSOCKET_PLUG_ART ]] && echo "$WEBSOCKET_PLUG_ART $OK" || echo "$WEBSOCKET_PLUG_ART $FAILURE" exit 1
env:
OK: 'exists'
FAILURE: 'does not exist'
DEVTOOLS_PLUG_ART: './@ngxs/devtools-plugin'
FORM_PLUG_ART: './@ngxs/form-plugin'
HMR_PLUG_ART: './@ngxs/hmr-plugin'
LOGGER_PLUG_ART: './@ngxs/logger-plugin'
ROUTER_PLUG_ART: './@ngxs/router-plugin'
STORAGE_PLUG_ART: './@ngxs/storage-plugin'
STORE_ART: './@ngxs/store'
WEBSOCKET_PLUG_ART: './@ngxs/websocket-plugin'
43 changes: 43 additions & 0 deletions .github/actions/download-integration-test-artifacts/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# For more information see:
# - https://docs.github.com/en/actions/creating-actions/creating-a-composite-action
# - https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#outputsoutput_id

# The action can be referenced in workflows like:
# - ngxs/store/.github/actions/download-integration-test-artifacts@master
# - ./.github/actions/download-integration-test-artifacts

name: download-integration-test-artifact
description: Downloads all integration test artifacts with names such as 'hello-world-ng11-ivy'.

inputs:
path:
description: A path to download the artifacts.
required: true
default: './integrations'

runs:
using: 'composite'
steps:
- name: Download hello-world-ng11-ivy artifacts
uses: actions/download-artifact@v3
with:
name: hello-world-ng11-ivy
path: ${{ inputs.path }}

- name: Download hello-world-ng12-ivy artifacts
uses: actions/download-artifact@v3
with:
name: hello-world-ng12-ivy
path: ${{ inputs.path }}

- name: Download hello-world-ng13-ivy artifacts
uses: actions/download-artifact@v3
with:
name: hello-world-ng13-ivy
path: ${{ inputs.path }}

- name: Download hello-world-ng14-ivy artifacts
uses: actions/download-artifact@v3
with:
name: hello-world-ng14-ivy
path: ${{ inputs.path }}
66 changes: 66 additions & 0 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# For more information see:
# - https://docs.github.com/en/actions/creating-actions/creating-a-composite-action
# - https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#outputsoutput_id

# The action can be referenced in workflows like:
# - ngxs/store/.github/actions/setup@master
# - ./.github/actions/setup

name: setup
description: Set up environment

inputs:
node-version:
description: Node version
required: true
default: 16.x
github-ref-name:
description: Branch name
required: true
default: ${{ github.ref_name }}
github-sha:
description: Commit hash
required: true
default: ${{ github.sha }}

runs:
using: 'composite'
steps:
- name: Don't save Bash session history
shell: bash
run: unset HISTFILE

- name: Use Node.js ${{ inputs.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ inputs.node-version }}
check-latest: true

- name: Configure kernel (increase watchers)
shell: bash
run: echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

- name: Workspace cache
uses: actions/cache@v3
id: workspace-cache
env:
cache-name: workspace-cache
with:
path: |
~/.cache
./node_modules
./@ngxs
./integrations/hello-world-ng11-ivy/dist-integration
./integrations/hello-world-ng12-ivy/dist-integration
./integrations/hello-world-ng13-ivy/dist-integration
./integrations/hello-world-ng14-ivy/dist-integration
key: ${{ runner.os }}-node-${{ inputs.node-version }}-yarn-${{ hashFiles('**/yarn.lock') }}-branch-${{ inputs.github-ref-name }}-sha-${{ inputs.github-sha }}
restore-keys: |
${{ runner.os }}-node-${{ inputs.node-version }}-yarn-${{ hashFiles('**/yarn.lock') }}-branch-${{ inputs.github-ref-name }}-sha-
${{ runner.os }}-node-${{ inputs.node-version }}-yarn-${{ hashFiles('**/yarn.lock') }}-branch-
${{ runner.os }}-node-${{ inputs.node-version }}-yarn-
${{ runner.os }}-

- name: Install project dependencies
shell: bash
run: yarn install --frozen-lockfile --non-interactive
34 changes: 34 additions & 0 deletions .github/actions/upload-integration-test-artifact/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# For more information see:
# - https://docs.github.com/en/actions/creating-actions/creating-a-composite-action
# - https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#outputsoutput_id

# The action can be referenced in workflows like:
# - ngxs/store/.github/actions/upload-integration-test-artifact@master
# - ./.github/actions/upload-integration-test-artifact

name: upload-integration-test-artifact
description: Upload an integration test artifact with a name such as 'hello-world-ng11-ivy'.

inputs:
script:
description: A script with a name such as 'integration:ng11:ivy' that generates an integration test artifact.
required: true

runs:
using: 'composite'
steps:
- name: Generate an artifact name
id: artifact-name
shell: bash
run: |
echo "Replace colons with dashes, substring 'integration' with a substring 'hello-world'. Example result: hello-world-ng11-ivy"
echo "::set-output name=value::$(echo ${SCRIPT} | sed -r "s/:/-/g" | sed -r "s/integration/hello-world/g")"
env:
SCRIPT: ${{ inputs.script }}

- name: Upload ${{ inputs.script }} artifact
uses: actions/upload-artifact@v3
with:
name: ${{ steps.artifact-name.outputs.value }}
path: ./integrations/**/dist-integration/main.*.js
retention-days: 1
146 changes: 146 additions & 0 deletions .github/workflows/pr-validation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# For more information see:
# - https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
# - https://docs.github.com/en/actions/learn-github-actions/contexts#github-context

name: pr-validation

on:
pull_request:

defaults:
run:
shell: bash

concurrency:
group: ${{ github.head_ref }}.${{ github.sha }}.pr-validation
cancel-in-progress: true

jobs:
premerge-build:
runs-on: ubuntu-latest

steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
fetch-depth: 0
Comment on lines +25 to +26
Copy link
Member

@markwhitfeld markwhitfeld Jun 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a good default. Is there a reason why this is not done in the other workflows for the checkout step?
If they can all be the same, then this could actually be moved into the setup composite action.

Copy link
Contributor Author

@rfprod rfprod Jun 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@markwhitfeld fetch-depth: 0 stands for all history for all branches and tags. It should be a bit slower and it pulls much more data than needed for other workflows than those that validate pull requests. From my experience with common repos and monorepos, it is generally not needed to pull all git history in the trunk or release workflows, because all tests that might require full commit history of all branches and tags (e.g. validate commit messages using commitlint) were done earlier during the corresponding PR validation.

If you think the default fetch-depth: 0 makes sense, so be it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Makes sense. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@markwhitfeld one more thing about the actions/checkout action.

Sources should be checked out prior to any steps that use composite actions like ./github/actions/..., because relative references requires already checkout out sources.
If you want to wrap the step that check out sources as a separate composite action, the only way to reference this composite action that will work, if I understand correctly, is ngxs/store/.github/actions/...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, hahaha. Yes, that makes total sense!

ref: ${{ github.event.pull_request.head.sha }}

- name: Setup
uses: ./.github/actions/setup
with:
node-version: 16.x
github-ref-name: ${{ github.ref_name }}
github-sha: ${{ github.event.pull_request.head.sha }}

- name: Build NGXS
run: yarn build

premerge-test:
runs-on: ubuntu-latest
needs: premerge-build

strategy:
matrix:
node-version: [16.x]
script: [lint, eslint, 'test:ci']

steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.sha }}

- name: Setup
uses: ./.github/actions/setup
with:
node-version: ${{ matrix.node-version }}
github-ref-name: ${{ github.ref_name }}
github-sha: ${{ github.event.pull_request.head.sha }}

- name: Run ${{ matrix.script }}
run: yarn ${{ matrix.script }}

- name: Upload coverage results to Code Climate
if: ${{ matrix.script == 'test:ci' }}
run: |
curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > /tmp/cc-test-reporter
chmod +x /tmp/cc-test-reporter
/tmp/cc-test-reporter after-build --coverage-input-type lcov --exit-code 0
env:
CC_TEST_REPORTER_ID: 3f4c9a9d57ded045e0f9ab5d23e5bbcbf709bb85637bea555f1233e72134b818 # TODO: better store it in the repository secrets. Name the variable: CC_TEST_REPORTER_ID. Then delete this line and uncomment the next one
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, can we move this to secrets

# CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}

rfprod marked this conversation as resolved.
Show resolved Hide resolved
premerge-integration-test:
runs-on: ubuntu-latest
needs: premerge-build

strategy:
matrix:
node-version: [16.x]
script:
- 'integration:ng7'
- 'integration:ng8'
- 'integration:ng9:ivy:off'
- 'integration:ng9:ivy'
- 'integration:ng10:ivy:off'
- 'integration:ng10:ivy'
- 'integration:ng11:ivy:off'
- 'integration:ng11:ivy'
- 'integration:ng12:ivy'
- 'integration:ng13:ivy'
- 'integration:ng14:ivy'
- 'test:types'

steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.sha }}

- name: Setup
uses: ./.github/actions/setup
with:
node-version: ${{ matrix.node-version }}
github-ref-name: ${{ github.ref_name }}
github-sha: ${{ github.event.pull_request.head.sha }}

- name: Run ${{ matrix.script }}
run: yarn ${{ matrix.script }}

- name: Upload an integration test artifact
if: ${{ matrix.script == 'integration:ng11:ivy' || matrix.script == 'integration:ng12:ivy' || matrix.script == 'integration:ng13:ivy' || matrix.script == 'integration:ng14:ivy' }}
uses: ./.github/actions/upload-integration-test-artifact
with:
script: ${{ matrix.script }}

premerge-bundlesize:
runs-on: ubuntu-latest
needs: premerge-integration-test

strategy:
matrix:
node-version: [16.x]
script: [bundlesize]
markwhitfeld marked this conversation as resolved.
Show resolved Hide resolved

steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.sha }}

- name: Setup
uses: ./.github/actions/setup
with:
node-version: ${{ matrix.node-version }}
github-ref-name: ${{ github.ref_name }}
github-sha: ${{ github.event.pull_request.head.sha }}

- name: Download integration test artifacts
uses: ./.github/actions/download-integration-test-artifacts

- name: Run ${{ matrix.script }}
run: yarn ${{ matrix.script }}
Loading