2.1 Test runs

Start and end of a test run

At the start, during and at the end the test the Gatling demo script will send calls to the targets-io dashboard. If a test run is stopped before the specified end of the test (see application.conf) the test run is marked as incomplete. You can see incomplete test run by unchecking the "Show completed test runs only" checkbox.

An incomplete test run can be marked as complete via the test run menu.

It is also possible to add a test run manually via the dashbboard menu. The following values have to be entered:

• Start (required)
• End (required)
• Product (required)
• Product release
• Dashboard (required)
• Test run ID (required)
• CI Build result URL: link to build result url in your CI server
• Annotations

Annotations

You can add annotations to a test run via "Annotate test run" in the test run menu. When annotations are made for a test run a "information" icon will show up in the test run overview for that test run. You can view and edit the annotations by clicking this icon.

Benchmarking

If benchmarking is enabled for a dashboard, the test run view has has the following columns:

• Test run ID: unique ID for the test run
• Start test run
• End Test run
• Requirements: shows the consolidated requirements results for the test run.
• Previous build: shows the consolidated previous build benchmark results for the test run.
• Fixed baseline: shows the consolidated fixed baseline benchmark results for the test run.
• Summary: the test run summary of this test run (if created)

The fixed baseline for the dashboard is marked and it's row row is highlighted. You can set a different test run as fixed baseline from the test run menu at the end of the row.

The test run menu has the following options:

• Update benchmarks: this triggers the dashboard to redo all the benchmarks for this test run.
• Set as baseline: sets the test run as fixed baseline and triggers to redo a fixed baseline benchmark for all test runs executed after this test run
• View in CI server: if the test run was executed from Jenkins, this link will take you to the build result page in Jenkins
• Mark as complete: for incomplete test runs only.
• Set product release
• Edit test run
• Annotate test run
• Delete test run: this will delete the test run including linked events.

To drill down into detailed benchmark results, click on the results icon. This will take to you to the detailed benchmark results view.

This view presents you the benchmark results for each metric individually, for both the passed as the failed. If you click on the result icon you will be linked to the graph for the metric to do a graphical inspection of the results.

REST API

The test run benchmark results are also available via a REST API:

http://<targets-io host>/benchmarks/<product name>/<dashboard name>/<test run ID>

for example:

http://targets-io-host:3000/benchmarks/GATLING/NIGHTLY/DEMO-GATLING-BENCHMARKING-MULTIJOB-25

The response contains the consolidated test run benchmark results that can be used to assert the test results using Gatling to make a Jenkins build pass or fail. See demo script

{ 
   "meetsRequirement": true, 
  "benchmarkResultPreviousOK": false, 
  "benchmarkResultFixedOK": true 
}