exercise-3

Exercise 3: Using the wdio test runner

There are two methods of running your WebDriver.io tests, and they are well documented in the official docs called "Standalone vs WDIO". Standalone mode is fine for first demos, or building your own entire framework, but using WDIO is the preferred method of running e2e tests at scale.

The WDIO Testrunner http://v4.webdriver.io/guide/getstarted/modes.html#The-WDIO-Testrunner

"The main purpose of WebdriverIO though is end to end testing on big scale. We therefore implemented a test runner that helps you to build a reliable test suite that is easy to read and maintain. The test runner takes care of many problems you are usually facing when working with plain automation libraries.

For one it organizes your test runs and splits up test specs so your tests can be executed with maximum concurrency. It also handles session management and provides a lot of features that help you to debug problems and find errors in your tests."

That being said, we'll use the WDIO test runner from this point forward!

Overview

• Success criteria
• Instructions
  • Init the exercise
  • Using the wdio command
  • Create a configuration file
  • Customizing the configuration
  • Create a test spec
  • Running wdio
  • First passing test
  • Launch wdio with an npm script
• What we learned
• Up next

---

Success criteria

1. Understand the difference between the standalone mode and using the WDIO test runner.
2. Setup your test runner to use wdio, the preferred way to run your e2e tests.
3. Add a convenient npm script

Instructions

Init the exercise

1. Change to the correct directory

cd exercise-3

2. Install dependencies

npm i

Using the wdio command

The wdio package is already installed with webdriverio, and available as an executable in the node_modules directory. Let's test that it's working:

./node_modules/.bin/wdio --help

If it's working, you'll see the help docs printed to the console:

$ ./node_modules/.bin/wdio --help
WebdriverIO CLI runner

Usage: wdio [options] [configFile]
Usage: wdio config
Usage: wdio repl [browserName]
config file defaults to wdio.conf.js
The [options] object will override values from the config file.
An optional list of spec files can be piped to wdio that will override configured specs.

Options:
(...)

Create a configuration file

The wdio command can help us setup our initial configuration file by walking us through some questions which will automatically scaffold your tests appropriately.

1. Run the wdio command to be asked some questions:

./node_modules/.bin/wdio

Note: See the screen shots below. You'll accept the defaults on many of the questions by simply pressing the enter key.

2. When asked where your test specs should be located, let's choose a slightly different path that is more selective about the test spec file globs.

Instead, enter ./tests/browser/**/*.spec.js:

Where are your test specs located? (./test/specs/**/*.js) ./tests/browser/**/*.spec.js

3. When asked which reporter, choose dot reporter. (It will be chosen by default if you don't make a selection.)

4. When asked which services, use the arrow keys and spacebar to scroll and select these options:

Choosing the selenium-standalone service means that we don't have to manually run our Selenium server, wdio will automatically start that service up for us.

This also makes running the service more consistent and reliable with multiple developers.

5. Choose silent for logging verbosity. We don't need all the extra noise quite yet.

6. When asked for the base url, enter our example app's url (without the trailing slash): http://localhost:3000

A successful installation

And you'll see the new devDependencies which were automatically added to your package.json:

"devDependencies": {
  "wdio-dot-reporter": "0.0.10",
  "wdio-mocha-framework": "^0.6.4",
  "wdio-selenium-standalone-service": "0.0.10",
  "webdriverio": "^4.14.0"
}

Customizing the configuration

1. First, let's make sure the correct test files are running. We'll exclude the solution test file by adding the appropriate path to the exclude property:

specs: [
    './tests/browser/**/*.spec.js'
],
// Patterns to exclude.
exclude: [
    './tests/browser/solution/**/*.spec.js'
],

Note: If you have problems getting your tests to run in any of the exercises, or simply want to skip ahead, you can change the specs path to only run the solution files, as shown below:

// Only run the solution test files 
specs: [
    // './tests/browser/**/*.spec.js'
    './tests/browser/solution/**/*.spec.js'
],
exclude: [
    // './tests/browser/solution/**/*.spec.js'
],

2. As we did in the previous exercise, let's change the browserName default from firefox to chrome.

Look for the capabilities object in your wdio.conf.js file:

capabilities: [{
    // maxInstances can get overwritten per capability. So if you have an in-house Selenium
    // grid with only 5 firefox instances available you can make sure that not more than
    // 5 instances get started at a time.
    maxInstances: 5,
    //
    browserName: 'chrome'
}],

3. Since our first test spec (example.spec.js) is still using promises, let's sync to false for now.

// By default WebdriverIO commands are executed in a synchronous way using
// the wdio-sync package. If you still want to run your tests in an async way
// e.g. using promises you can set the sync option to false.
sync: false,

Create a test spec

Create a new test spec file called example.spec.js in exercise-3/tests/browser, and paste this log statement:

console.log('browser.desiredCapabilities', browser.desiredCapabilities);

This where we will eventually see our tests execute, but for now this simply logs the global browser object so we know it's working. It also gives us some insights into the global browser object in WebDriver.io.

Running wdio

We can now execute our wdio test runner, though it's not going to give us any passing tests quite yet:

./node_modules/.bin/wdio

You should see the wdio command execute successfully, using the wdio.conf.js file for configuration, executing our exercise-3/tests/browser/example.spec.js test file where we've logged some global browser info to the console.

Note that none of the tests are passing, because we haven't really created our next set of tests quite yet ...

First passing test

Next we're going to convert our previous test from example-2 to use wdio.

Since wdio does much of the setup and teardown of tests, we can delete the require() and options lines, as well as calls to .remote(), .init(), and .end(). Additionally, the global WebDriver client is a variable called browser, so we'll update that, too.

And because the base url is defined in our configuration file, we can simply point to the root path. So we'll go ahead and delete those obsolete lines, change webdriverio to browser, and update the url. You'll also see that we've wrapped the browser code in Mocha's describe() and it() functions so that we see passing tests:

1. Open the existing tests/browser/example.spec.js and replace it with the following code (based on the previous exercise):

/**
 * example.spec.js
 */
var assert = require('assert');

describe('Example', function () {
    describe('page title', function () {
        var expectedPageTitle = "Verbo - Simple travel planning";

        it('should be "' + expectedPageTitle + '"', function () {
            // Navigate to the home page.
            return browser
                .url('/')
                .getTitle().then(function (title) {
                    // Log the page title.
                    console.log('🤖 The page title is: ' + title);

                    // Assert the homepage title is what we expect it to be.
                    assert.equal(title, expectedPageTitle);

                    // Want to see what a failure looks like?  Uncomment the line below to try the `.not` syntax!
                    // assert.notEqual(title, expectedPageTitle);
                });
        });
    });
});

2. Run the test again, and you should see something like this, with 1 test passing:

./node_modules/.bin/wdio

You can also try intentionally causing a test to fail to see what that looks like. Uncomment the assert.notEqual() statement to see a test failure:

Launch wdio with an npm script

Let's add an npm run [command] script so we can avoid calling wdio directly, and make it easier for other developers and build tools to run your e2e test suite.

You can call your script just about anything, but at HomeAway, we typically have a pattern where a variety of "sub-scripts" use the colons to group similar scripts (e.g., npm run test:browser). Note that the npm run test command is more typically used to run unit tests.

1. Open package.json
2. Notice the scripts field:

"scripts": {
    "test:browser": "wdio"
},

3. Now, simply run npm run test:browser from now on to run your suite of tests. Notice the > wdio in the output:

Pro tip: Another benefit to using npm scripts is that they already search node_modules/.bin for local executables, so we can simple execute wdio without the rest of the path.

✅ Congratulations! With the wdio test runner setup, you're getting closer to a fully configured environment for running e2e tests.

---

What we learned

• You should now understand the difference between the standalone mode and using the WDIO test runner, and why WDIO is preferred.
• You can now run wdio tests, using the configuration file you created.
• You wrote your first simple test spec!
• We created an npm run test:browser script that will make running wdio easier in the future.

Up next

Exercise 4: Adding mocha and chai