Tests

Tests are implemented using JUnit. Tests for classes in a bundle are not added to the bundle itself, but to a separate bundle for fragment that adds the suffix .test to the name of the original bundle.

Names of test classes by convention end with Test, or with IT if they are integration tests.

Test bundles/fragments need to be added to the Tests.product. Make sure the product validates.

Please see the sections below in how to run the tests. You can find more details on the arguments for the test runner here.

Usually when running the tests like this you should at the end get a summary and details on the failed tests. It was observed that this does not work in all cases, which prevents you from easily getting the information on if and what tests failed. As a workaround you can add the argument -out <file> to create a JUnit test report XML file. You can check the file for the results or visualize it, e.g. in this online tool.

Run all tests

To run all tests the simplest way is to run the Tests.product from within Eclipse.

Run a single test class or method

To run single test classes or methods, you can right click them in Eclipse and select Run As → JUnit Plug-in Test.

However with newer Eclipse versions startup times of tests run this way have been very long. What you can do to speed this process up is adapting the run configuration for the test, to only include required bundles and to only auto-start bundles where this is required.

This is a very tedious process if you need to repeat that for every test, so as an alternative you can also run individual test classses or methods based on the test product. This is the suggested approach:

1. If you want to run only certain tests with the test product you first need a run configuration that you can adapt. The simplest way to create one is to use the option to run the Tests.product from the Eclipse editor. You can cancel the run right afterwards.
2. Now you have a run configuration based on the test product that you can adapt. If you want to avoid changes to your adaptions when you run the product again, create a copy of the run configuration.
3. In the run configuration you can extend the program arguments by arguments supported by the test runner used in the test product to select individual test classes or methods:
   • Specify one or more test classes using the -class <pattern> arguments. The pattern can be a fully qualified class name or a simple class name and also allows * as a simple wild card.
   • Specify one or more test methods to run using the -method <method-name> arguments. If omitted all test methods in the identified classes will be run.
   • Specifying the bundle where the classes can be found can speed up looking for the classes. Use -bundle <bundle-symbolic-name> arguments to specify one ore more bundles to search in. If not specified test classes will be looked for in all bundles ending on .test.
4. Run or debug the test(s) based on your adapted run configuration. Please be aware that this does not work together with the JUnit integration in Eclipse, so you need to check the console output whether a test was run and if it was successful or failed.

Allure

Note: Integration of Allure in testing is not supported anymore in hale 5+ (additional effort would need to be spent to make it working again)

The Allure test framework can be used to augment tests with additional information like file attachments, categorisation in Features and Stories and partitioning into steps.

Generation of Allure test report files is currently only supported when running tests via the Tests product.

To use Allure features you have to annotate your code with Allure annotations. To that purpose, add the package ru.yandex.qatools.allure.annotations to the imported packages in your bundle.

To support @Step, @Attachment and @Parameter annotations AspectJ support is required. AspectJ is configured in the test product using Equinox Weaving, but additionally you need specific imports in your test bundle:

org.aspectj.lang
org.aspectj.runtime.reflect
ru.yandex.qatools.allure.aspects;apply-aspects:=true

Note the apply-aspects section. This informs Equinox Weaving that we want the aspects applied to our bundle. Together with the version information, the complete imports related to Allure in the test bundle then might look like this:

 org.aspectj.lang;version="1.8.3",
 org.aspectj.runtime.reflect;version="1.8.3",
 ru.yandex.qatools.allure.annotations;version="1.5.0",
 ru.yandex.qatools.allure.aspects;version="1.5.0";apply-aspects:=true