manual-e2e-run

.github/actions/frontend/runtime/e2e_pending_status/README.md

@@ -0,0 +1,25 @@
+# Frontend Runtime - E2E Pending Status
+
+This [composite action](./action.yml) is responsible for reporting that the E2E tests have begun and are pending via a status check in the PR, providing a quick feedback loop for developers.
+
+## Inputs
+
+This action takes the following inputs:
+
+| Name                        | Type    | Default                      | Required  | Description                                               |
+| --------------------------- | ------- | ---------------------------- | --------- | --------------------------------------------------------- |
+| `github_token`              | String  |                              | True      | Github access token
+
+## Outputs
+
+No outputs provided.
+
+## Example Usage
+
+```yaml
+steps:
+  - name: e2e_status
+    uses: jupiterone/.github/.github/actions/frontend/runtime/e2e_status
+    with:
+      github_token: ${{ secrets.GITHUB_TOKEN }}
+```

.github/actions/frontend/runtime/e2e_pending_status/action.yml

@@ -0,0 +1,24 @@
+name: Set the status of the PR when E2E tests have been triggered
+
+inputs:
+  github_token:
+    type: string
+    required: true
+
+runs:
+  using: "composite"
+  steps:
+    - name: e2e_pr_status_inputs
+      shell: bash
+      run: |
+        echo "github_token=${{ inputs.github_token }}"
+    - name: get_branch_of_pr
+      id: get_branch_of_pr
+      uses: xt0rted/pull-request-comment-branch@v2
+    - name: set_status_as_pending
+      uses: myrotvorets/[email protected]
+      with:
+        sha: ${{ steps.get_branch_of_pr.outputs.head_sha }}
+        token: ${{ inputs.github_token }}
+        status: pending
+        context: 'pr / E2E Status'

.github/actions/frontend/runtime/e2e_pending_status/mocks.ts

@@ -0,0 +1,9 @@
+/*
+This file is automatically leveraged when tests are run to determine which
+steps should be skipped in the composite action. If these steps were not
+mocked, they would break the test.
+*/
+export const E2E_PENDING_STATUS_MOCK_STEPS = [
+  { name: 'get_branch_of_pr' },
+  { name: 'set_status_as_pending' },
+];

.github/actions/frontend/runtime/e2e_status/README.md

@@ -1,6 +1,6 @@
 # Frontend Runtime - E2E Status
 
-This [composite action](./action.yml) is responsible for reporting the status of the E2E run. Specifically it is configurable to allow a PR to still be merged even if the E2E tests fail (if `e2e_pass_on_error: true`). By default this job is never reached if E2E tests fail and `e2e_pass_on_error` is set to `false`. 
+This [composite action](./action.yml) is responsible for reporting the status of the E2E run as both a comment on the PR as well as updating the status check. Specifically it is configurable to allow a PR to still be merged even if the E2E tests fail (if `e2e_pass_on_error: true`). By default this job is never reached if E2E tests fail and `e2e_pass_on_error` is set to `false`. 
 
 ## Inputs
 
@@ -9,7 +9,6 @@ This action takes the following inputs:
 | Name                        | Type    | Default                      | Required  | Description                                               |
 | --------------------------- | ------- | ---------------------------- | --------- | --------------------------------------------------------- |
 | `github_token`              | String  |                              | True      | Github access token
-| `e2e_passed`                | Boolean |                              | True      | The status of the e2e_run
 | `e2e_pass_on_error`         | Boolean | False                        | False     | Pass the workflow even if the E2E test fail. Should be defined as an input at the workflow level.
 
 ## Outputs
@@ -24,6 +23,5 @@ steps:
     uses: jupiterone/.github/.github/actions/frontend/runtime/e2e_status
     with:
       github_token: ${{ secrets.GITHUB_TOKEN }}
-      e2e_passed: ${{ needs.e2e_run.outputs.test_passed }}
       e2e_pass_on_error: ${{ inputs.e2e_pass_on_error }}
 ```

.github/actions/frontend/runtime/e2e_status/action.yml

@@ -4,9 +4,6 @@ inputs:
   github_token:
     type: string
     required: true
-  e2e_passed:
-    type: boolean
-    required: true
   e2e_pass_on_error:
     type: boolean
     required: false
@@ -19,19 +16,38 @@ runs:
       shell: bash
       run: |
         echo "github_token=${{ inputs.github_token }}"
-        echo "e2e_passed=${{ inputs.e2e_passed }}"
         echo "e2e_pass_on_error=${{ inputs.e2e_pass_on_error }}"
-    - name: pr_status
-      uses: Sibz/github-status-action@v1
-      continue-on-error: ${{ fromJSON(inputs.e2e_pass_on_error) }}
+    - name: add_result_as_pr_comment
+      uses: actions/github-script@v6
       with:
-        authToken: ${{ inputs.github_token }}
-        context: 'E2E Test Status'
-        description: ${{ (fromJSON(inputs.e2e_passed) && 'E2E Test Passed') || 'E2E Test Failed' }}
-        state: ${{ (fromJSON(inputs.e2e_passed) && 'success') || 'failures' }}
-        sha: ${{ github.sha }}
-    # We have to manually output an exit code of 0 to ensure the action passes if e2e_pass_on_error is true
-    - name: pass_with_failures
-      shell: bash
+        script: |
+          const name = 'pr / E2E Status';
+          const url = '${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}';
+          const success = '${{ inputs.job_status }}' === 'success';
+          const body = `${name}: ${success ? 'succeeded ✅' : 'failed ❌'}\n${url}`;
+
+          await github.rest.issues.createComment({
+            issue_number: context.issue.number,
+            owner: context.repo.owner,
+            repo: context.repo.repo,
+            body: body
+          })
+    - name: get_branch_of_pr
+      id: get_branch_of_pr
+      uses: xt0rted/pull-request-comment-branch@v2
+    - name: set_commit_status
+      uses: myrotvorets/[email protected]
+      with:
+        sha: ${{ steps.get_branch_of_pr.outputs.head_sha }}
+        token: ${{ inputs.github_token }}
+        status: ${{ job.status }}
+        context: 'pr / E2E Status'
+    # Manually set the commit status to success to allow the PR to be merged
+    - name: force_status_as_success
+      uses: myrotvorets/[email protected]
       if: ${{ fromJSON(inputs.e2e_pass_on_error) }}
-      run: exit 0
+      with:
+        sha: ${{ inputs.branch_sha }}
+        token: ${{ inputs.github_token }}
+        status: success
+        context: 'pr / E2E Status'

.github/actions/frontend/runtime/e2e_status/mocks.ts

@@ -4,5 +4,8 @@ steps should be skipped in the composite action. If these steps were not
 mocked, they would break the test.
 */
 export const E2E_STATUS_MOCK_STEPS = [
-  { name: 'pr_status' },
+  { name: 'add_result_as_pr_comment' },
+  { name: 'get_branch_of_pr' },
+  { name: 'set_commit_status' },
+  { name: 'force_status_as_success' },
 ];

.github/actions/frontend/runtime/e2e_status/test/action.test.ts

@@ -11,7 +11,7 @@ afterEach(async () => {
   await mockGithub.teardown();
 });
 
-test('pass_with_failures is not called when e2e_passed is false', async () => {
+test('force_status_as_success is not called when e2e_passed is false', async () => {
   mockGithub = new MockGithub(getCompositeActionConfig({ directory: __dirname, repoName, actionTriggeringComposite: 'action_test_fail.yml' }));
 
   await mockGithub.setup();
@@ -22,12 +22,12 @@ test('pass_with_failures is not called when e2e_passed is false', async () => {
     originDirectory: __dirname
   });
 
-  const result = getTestResult({ results, name: 'pass_with_failures' });
+  const result = getTestResult({ results, name: 'force_status_as_success' });
 
   expect(result).toBeUndefined();
 });
 
-test('e2e_pass_on_error allows pass_with_failures to be called when e2e_passed is false', async () => {
+test('e2e_pass_on_error allows force_status_as_success to be called when e2e_passed is false', async () => {
   mockGithub = new MockGithub(getCompositeActionConfig({ directory: __dirname, repoName, actionTriggeringComposite: 'action_test_pass.yml' }));
 
   await mockGithub.setup();
@@ -38,7 +38,7 @@ test('e2e_pass_on_error allows pass_with_failures to be called when e2e_passed i
     originDirectory: __dirname
   });
 
-  const result = getTestResult({ results, name: 'pass_with_failures' });
+  const result = getTestResult({ results, name: 'force_status_as_success' });
 
   expect(result).not.toBeUndefined();
 });

.github/actions/frontend/runtime/magic_url/README.md

@@ -21,9 +21,10 @@ No outputs provided.
 ```yaml
 permissions:
   id-token: write
-  contents: read
-  pull-requests: write
   statuses: write
+  contents: read
+  issues: read
+  pull-requests: read
 steps:
   - name: magic_url
     uses: jupiterone/.github/.github/actions/frontend/runtime/magic_url

.github/actions/frontend/runtime/magic_url/action.yml

@@ -36,12 +36,14 @@ runs:
       # This bucket file location is static and editing it will break the Magic URL. This pushes the entire directory which includes the bundle and remote types if applicable
       run: |
         aws s3 sync deploy/dist s3://apps.dev.jupiterone.io/static/manual-deploy/${{ github.event.repository.name }}@${{ inputs.migration_number }}/PR-${{ github.event.number }}/
+    - name: get_branch_of_pr
+      id: get_branch_of_pr
+      uses: xt0rted/pull-request-comment-branch@v2
     - name: show_magic_url_in_pr
-      uses: Sibz/github-status-action@v1
-      with: 
-        authToken: ${{ inputs.github_token }}
-        state: 'success'
+      uses: myrotvorets/[email protected]
+      with:
+        sha: ${{ steps.get_branch_of_pr.outputs.head_sha }}
+        token: ${{ inputs.github_token }}
+        status: success
         context: 'Magic URL'
-        description: "Use the 'Details' link to view this PR in dev"
-        target_url: https://apps.dev.jupiterone.io${{ inputs.magic_url_route }}?magic2=%7B%22${{ github.event.repository.name }}%40${{ inputs.migration_number }}%22:%22PR-${{ github.event.pull_request.number }}%22%7D
-        sha: ${{github.event.pull_request.head.sha || github.sha}}
+        targetUrl: https://apps.dev.jupiterone.io${{ inputs.magic_url_route }}?magic2=%7B%22${{ github.event.repository.name }}%40${{ inputs.migration_number }}%22:%22PR-${{ github.event.pull_request.number }}%22%7D

.github/actions/frontend/runtime/magic_url/mocks.ts

@@ -7,5 +7,6 @@ export const MAGIC_URL_MOCK_STEPS = [
   { name: 'npm_build' },
   { name: 'configure_aws_credentials' },
   { name: 'deploy_artifacts_to_s3' },
+  { name: 'get_branch_of_pr' },
   { name: 'show_magic_url_in_pr' },
 ];

.github/workflows/docs/frontend/frontend_npm_pr.md

@@ -1,6 +1,6 @@
 # Default flow for an NPM package when a PR is opened
 
-This is the [default workflow](../../frontend_npm_pr.yml@v#) that is run when a `PR is opened` for an `npm package`. It is meant to test the quality and safety of the code being committed.
+This is the [default workflow](../../frontend_npm_pr.yml) that is run when a `PR is opened` for an `npm package`. It is meant to test the quality and safety of the code being committed.
 
 ## Inputs
 

.github/workflows/docs/frontend/frontend_npm_release.md

@@ -1,6 +1,6 @@
 # Default flow for an NPM package when its merged to mai
 
-This is the [default workflow](../../frontend_npm_release.yml@v#) that is run when a `PR is merged to main` for an `npm package`. It is meant to release the package and capture information in Cortex related to the package.
+This is the [default workflow](../../frontend_npm_release.yml) that is run when a `PR is merged to main` for an `npm package`. It is meant to release the package and capture information in Cortex related to the package.
 
 ## Inputs
 

.github/workflows/docs/frontend/frontend_runtime_application_manual_e2e_run.md

@@ -0,0 +1,84 @@
+# Default workflow for running E2E tests when a PR comment is left
+
+
+This is the [default workflow](../../frontend_runtime_application_manual_e2e_run.yml) that is leveraged to trigger E2E tests for for an `application runtime`. Specifically, when a PR is opened and a comment of `/run-e2e-tests` is left, it will trigger an E2E test run.
+
+## Inputs
+
+This action takes the following inputs:
+
+| Name                        | Type    | Default                      | Required  | Description                                                                            |
+| --------------------------- | ------- | ---------------------------- | --------- | -------------------------------------------------------------------------------------- |
+| `fallback_runner`            | String  | False                        | False      | If true will leverage ubuntu-latest, otherwise will fall back to the J1 in-house runner
+| `use_magic_url`              | Boolean | True                         | False      | Deploy to dev via a query param, required for normal SPAs
+| `use_e2e`                    | Boolean | False                        | False      | Run E2E test, in most case we want this
+| `e2e_filter_tags`            | String  |                              | False      | Tests will be filtered based on the tags defined here
+| `e2e_containers`             | String  | '["1"]'                      | False      | The number of tests that you want Cypress to run in parallel. For example to run your tests via 3 containers you would set the value to the following: '["1", "2", "3"]'.
+| `e2e_pass_on_error`          | Boolean | False                        | False      | Pass the workflow even if the E2E test fail
+| `e2e_artemis_config_path`    | String  | cypress/artemis-config.yaml  | False      | Used to determine the path to the artemis config file
+| `spec_to_run`                | String  | cypress/e2e/**/*.feature     | False      | Used to determine which test to run
+| `magic_url_route`            | String  | '/'                          | False      | The relative route the magic url should go to
+| `auto_cancel_after_failures` | Number  | 2                            | False      | Cancel the tests after this many failures
+
+## Secrets
+
+This action takes the following secrets:
+
+| Name                        | Required  | Description                               |
+| --------------------------- | --------- | ----------------------------------------- |
+| `NPM_TOKEN`                 | True      | A J1 npm.com Publish token
+| `CYPRESS_RECORD_KEY`        | True      | The record key associated with the project in Cypress.
+| `CYPRESS_PROJECT_ID`        | True      | The project ID associated with the project in Cypress
+| `CYPRESS_PASSWORD`          | False     | The password of the E2E username
+| `DOCKER_HUB_SRE`            | True      | The password to login to docker hub sre
+
+## Example Usage
+
+### Default Flow
+
+#### Usage
+
+```yaml
+# Triggered by adding a comment to a PR
+on:
+  issue_comment:
+    types: [created]
+
+jobs:
+  e2e_run:
+    # Check if the comments come from pull request and contains '/run-e2e-test'
+    if: github.event.issue.pull_request && contains(github.event.comment.body, '/run-e2e-tests')
+    uses: jupiterone/.github/.github/workflows/frontend_runtime_application_manual_e2e_run.yml@v#
+    secrets:
+      NPM_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}
+      DOCKER_HUB_SRE: ${{ secrets.DOCKER_HUB_SRE }}
+      CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
+      CYPRESS_PROJECT_ID: ${{ secrets.CYPRESS_PROJECT_ID }}
+```
+
+#### Diagram
+
+```mermaid
+graph LR;
+    A[start flow];
+
+    B[migration_number];
+    C[e2e_pending_status];
+    D[e2e_prepare];
+
+    E[magic_url];
+
+    F[e2e_run];
+    G[e2e_status];
+
+    A --> B;
+    A --> C;
+    A --> D;
+
+    B --> E;
+    D --> F;
+    E --> F;
+
+    F --> G;
+    C --> G;
+```