Skip to content

Commit

Permalink
chore: removing saucelabs support (#1077)
Browse files Browse the repository at this point in the history
Co-authored-by: Chunwai Li <[email protected]>
  • Loading branch information
patrickhousley and cwli24 authored Jun 21, 2024
1 parent ae2293d commit f5b9bc9
Show file tree
Hide file tree
Showing 75 changed files with 2,726 additions and 10,808 deletions.
3 changes: 1 addition & 2 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ module.exports = {
'tests/assets/modular/js-errors/js/vendor/**/*',
'tests/dts/**/*',

// Ignore JIL code since it is being replaced with WDIO
'tools/jil/**/*',
// Ignore old JIL test code since they are being migrated to WDIO
'tests/browser/**/*',
'tests/functional/**/*'
],
Expand Down
3 changes: 1 addition & 2 deletions .github/actions/post-release-updates/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,9 @@ await spawnAsync(
DEFAULT_SPAWN_OPTIONS
)

console.log('Updating SauceLabs browsers lists')
console.log('Updating LambdaTest browsers lists')
await spawnAsync(
`npm${os.platform() === 'win32' ? '.cmd' : ''}`,
[ 'run', 'sauce:get-browsers' ],
['run', 'lt:update-browsers'],
DEFAULT_SPAWN_OPTIONS
)
Expand Down
134 changes: 0 additions & 134 deletions .github/workflows/jil-all-browsers.yml

This file was deleted.

79 changes: 0 additions & 79 deletions .github/workflows/jil-single-browser.yml

This file was deleted.

36 changes: 0 additions & 36 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,42 +14,6 @@ jobs:
browser-target: chrome@latest
secrets: inherit

jil-smoke-unit:
name: 'JIL: Smoke Unit Test'
needs: wdio-smoke
uses: ./.github/workflows/jil-single-browser.yml
with:
browser-target: chrome@latest
collection: unit
secrets: inherit

jil-smoke-functional:
name: 'JIL: Smoke Functional Test'
needs: wdio-smoke
uses: ./.github/workflows/jil-single-browser.yml
with:
browser-target: chrome@latest
collection: functional
secrets: inherit

jil-polyfill-unit:
name: 'JIL: Polyfill Unit Test'
needs: jil-smoke-unit
uses: ./.github/workflows/jil-single-browser.yml
with:
browser-target: chrome@latest
collection: unit
secrets: inherit

jil-polyfill-functional:
name: 'JIL: Polyfill Functional Test'
needs: jil-smoke-functional
uses: ./.github/workflows/jil-single-browser.yml
with:
browser-target: chrome@latest
collection: functional
secrets: inherit

verify-ab-assets:
name: Verify A/B Assets
timeout-minutes: 60
Expand Down
12 changes: 1 addition & 11 deletions .github/workflows/pull-request-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -174,20 +174,10 @@ jobs:
latest-only: ${{ inputs.latest-only }}
secrets: inherit

jil:
name: JIL
if: github.event_name == 'workflow_dispatch'
needs: [find-pull-request]
uses: ./.github/workflows/jil-all-browsers.yml
with:
ref: 'refs/pull/${{ fromJSON(needs.find-pull-request.outputs.pull-request-target).pr_number }}/merge'
latest-only: ${{ inputs.latest-only }}
secrets: inherit

complete-status-comment-pull-request:
name: Comment pull request
if: always() && github.event_name == 'workflow_dispatch' && needs.find-pull-request.result == 'success'
needs: [find-pull-request,wdio-coverage,wdio,jil]
needs: [find-pull-request,wdio-coverage,wdio]
runs-on: ubuntu-latest
defaults:
run:
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ for more details).
## Testing

All changes to the Agent should include test coverage, and each PR must pass
all tests against all browsers in SauceLabs locally before being merged.
all tests against all browsers in LambdaTest locally before being merged.

We run tests on a variety of browsers and platforms to ensure that the agent runs safely for all users. We use Saucelabs, and the test matrix is defined [here](tools/jil/util/browsers.json).
We run tests on a variety of browsers and platforms to ensure that the agent runs safely for all users. We use LambdaTest.

When you first submit your PR, the tests will not be run automatically. After we review the PR, we will add a label that will trigger the full-matrix testing.

Expand Down
31 changes: 11 additions & 20 deletions DEVELOPING.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,7 @@ See the sections below for details about local and PR testing.

### Installing

The Browser agent uses a tool called the JavaScript Integration test Loader (`jil`) to run
tests (located in `/tools/jil`).
The Browser agent uses [webdriverio](https://webdriver.io/) (located in `/tools/wdio`) to run tests.

_Before running tests locally, be sure to [install and build](#building) from the root directory to ensure all dependencies are loaded and the application is properly built._

Expand All @@ -122,54 +121,46 @@ npm run build:all

### Running the test suite

To run all tests on a specific browser/platform, you can either run on Saucelabs or point the testing framework to your own Selenium server.

To run tests on Saucelabs, you will need your own Saucelabs account. Export your Saucelabs username and access key in these environment variables - SAUCE_USERNAME, SAUCE_ACCESS_KEY. After that you can use the following command to run tests on a specific browser. Note that the browser/platform needs to be defined in this [matrix file](tools/jil/util/browsers.json).
To run tests on LambdaTest, you will need your own LambdaTest account. Export your LambdaTest username and access key in these environment variables - LT_USERNAME, LT_ACCESS_KEY. After that you can use the following command to run tests on a specific browser. Note that the browser/platform needs to be defined in this [matrix file](tools/jil/util/browsers.json).

Here is an example of running all tests on the latest version of Chrome.

```
npm run test -- -s -b chrome@latest
```

Here is an example of using your own Selenium server:

```
npm run test -- -b chrome@latest --selenium-server=localhost:4444
npm run wdio -- -T -b chrome@latest
```

### Supported Browsers

- The browser agent is tested against this [list of browsers and environments](./tools/jil/util/browsers-supported.json). Use of the browser agent with untested browsers may lead to unexpected results.
- The browser agent is tested against the list of browsers found in `/tools/browsers-lists`. Use of the browser agent with untested browsers may lead to unexpected results.

**Important Notes:**

- `jil` does not handle building the agent automatically;
- `wdio` does not handle building the agent automatically;
either run `npm run build:all` after each change, or use `npm run watch` to automatically rebuild on each change.
- To pass arguments to the testing suite using `npm run test` you must separate your arguments from the npm script using an empty `--` parameter as was exemplified above.

### Running a single test

To run a single test in isolation, pass the path to `jil`:
To run a single test in isolation, pass the path to `wdio`:

```
npm run test -- tests/functional/api.test.js
npm run wdio -- tests/specs/api.test.js
```

### Debugging tests

To debug a unit test (`/tests/browser`) or the asset under test in a unit or functional test (`/tests/assets`), run the command below:
To debug the asset under test in a functional test (`/tests/assets`), run the command below:

```
npm run test-server
```

Running this command starts a server, available at http://localhost:3333, with a list of all available unit tests and test HTML pages with the Browser agent installed. Select a unit test from the list to run the test itself in your browser, or select an asset from the list to debug.
Running this command starts a server, available at http://bam-test-1.nr-local.net:3333, with a list of all available unit tests and test HTML pages with the Browser agent installed. Select a unit test from the list to run the test itself in your browser, or select an asset from the list to debug.

**Important**: When running `jil-server` be sure to tell HTML files which Browser agent type you want by adding a `?loader=spa` to the `querystring`. Here's an example:
When running the test server, you can to tell HTML files which Browser agent type you want by adding a `?loader=spa` to the `querystring`. Here's an example:

```
http://localhost:3333/tests/assets/spa/fetch.html?loader=spa
http://bam-test-1.nr-local.net:3333/tests/assets/spa/fetch.html?loader=spa
```

| Agent type | querystring name |
Expand Down
Loading

0 comments on commit f5b9bc9

Please sign in to comment.