Skip to content

OLD Prototype Running Tests

Jump to bottom
Marc Philipp edited this page Feb 7, 2017 · 1 revision

Running JUnit 5 Tests

Table of Contents

IDE Support

At this stage there is no direct support for running JUnit 5 tests in IDEs. However, we provide two intermediate solutions so that you can go ahead and try out the prototype today. You can use the ConsoleRunner or execute JUnit 5 tests with a JUnit 4 style runner.

Build Support

Gradle

The JUnit team has developed a very basic JUnit5Plugin that lets you run JUnit 4/5 tests in Gradle builds. See build.gradle in the junit5-gradle-consumer project for an example of the plugin in action.

Enabling the JUnit5Plugin

To use the JUnit5Plugin, you first need to configure build.gradle as follows.

buildscript {
	repositories {
		maven { url 'https://oss.sonatype.org/content/repositories/snapshots' }
	}
	dependencies {
		classpath 'org.junit.prototype:junit-gradle:5.0.0-SNAPSHOT'
	}
}

apply plugin: 'org.junit.gen5.gradle'

Configuring the JUnit5Plugin

Once the JUnit5Plugin has been applied, you can configure it as follows.

Caution: These options are very likely to change as we continue to work on the prototype.

junit5 {
	version '5.0.0-SNAPSHOT'
	runJunit4 true
	matchClassName '.*Test'
	includeTag 'fast'
}

Setting runJunit4 to true instructs the JUnit5Plugin to run JUnit 4 based tests as well. However, you will still need to configure a testCompile dependency on JUnit 4 in your project similar to the following.

dependencies {
	testCompile('junit:junit:4.12')
}

If you supply a tag to the includeTag configuration method, the JUnit5Plugin will only run JUnit 5 based tests that are tagged accordingly via the @Tag annotation.

Using the JUnit5Plugin

Once the JUnit5Plugin has been applied and configured, you have a new junit5Test task at your disposal.

Invoking gradlew clean junit5Test (or gradlew clean check) from the command line will execute all tests within the project whose class names match the regular expression supplied via matchClassName.

Executing the junit5Test task in the junit5-gradle-consumer project results in output similar to the following:

:junit5Test

Test run finished after 50 ms
[         3 tests found     ]
[         2 tests started   ]
[         1 tests skipped   ]
[         0 tests aborted   ]
[         2 tests successful]
[         0 tests failed    ]

BUILD SUCCESSFUL

If a test fails, the build will fail with output similar to the following:

:junit5Test

Test failures (1):
  junit5:com.example.project.SecondTest:mySecondTest
    com.example.project.SecondTest#mySecondTest
    => Exception: 2 is not equal to 1 ==> expected:<2> but was:<1>

Test run finished after 50 ms
[         3 tests found     ]
[         3 tests started   ]
[         0 tests skipped   ]
[         0 tests aborted   ]
[         2 tests successful]
[         1 tests failed    ]

:junit5Test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':junit5Test'.
> Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

Note: the exit value corresponds to the number of tests failed.

Current Limitations of the JUnit5Plugin

  • Even though the build will fail if a test fails, the results will not be included in the test report generated by Gradle.
  • With runJunit4 set to true, the JUnit5Plugin takes over the responsibility of running JUnit 4 tests, and the check Gradle task will no longer depend on the Gradle test task. Consequently, the results for JUnit 4 tests will not be included in the test report generated by Gradle.

Maven

We have developed a very basic provider for Maven Surefire that lets you run JUnit 4/5 tests through mvn test. The junit5-maven-consumer demonstrates how to use it and can serve as a starting point.

Basic usage in pom.xml:

	... 
	<build>
		<plugins>
			...
			<plugin>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>2.19</version>
				<dependencies>
					<dependency>
						<groupId>org.junit.prototype</groupId>
						<artifactId>surefire-junit5</artifactId>
						<version>5.0.0-SNAPSHOT</version>
					</dependency>
				</dependencies>
			</plugin>
		</plugins>
	</build>
	...

Console Runner

The ConsoleRunner is a command-line Java application that lets you run JUnit 4/5 tests and prints out test executions and results to the console.

Here's an example of its output:

Test execution started. Number of static tests: 2
Engine started: junit5
Test started:     My 1st JUnit 5 test! 😎 [junit5:com.example.project.FirstTest#myFirstTest(java.lang.String)]
Test succeeded:   My 1st JUnit 5 test! 😎 [junit5:com.example.project.FirstTest#myFirstTest(java.lang.String)]
Test skipped:     mySecondTest [junit5:com.example.project.SecondTest#mySecondTest()]
                  => Exception:   Skipped test method [void com.example.project.SecondTest.mySecondTest()] due to failed condition
Engine finished: junit5
Test execution finished.

Test run finished after 29 ms
[         2 tests found     ]
[         1 tests started   ]
[         1 tests skipped   ]
[         0 tests aborted   ]
[         1 tests successful]
[         0 tests failed    ]

Options

Caution: These options are very likely to change as we continue to work on the prototype.

NAME
        ConsoleRunner - console test runner

SYNOPSIS
        ConsoleRunner [(-a | --all)] [(-C | --disable-ansi-colors)]
                [(-D | --hide-details)] [(-h | --help)]
                [(-n <classnameFilter> | --filter-classname <classnameFilter>)]
                [(-t <tagsFilter> | --filter-tags <tagsFilter>)...]
                [(-x | --enable-exit-code)] [--] [<arguments>...]

OPTIONS
        -a, --all
            Run all tests

        -C, --disable-ansi-colors
            Disable colored output (not supported by all terminals)

        -D, --hide-details
            Hide details while tests are being executed. Only show the summary
            and test failures.

        -h, --help
            Display help information

        -n <classnameFilter>, --filter-classname <classnameFilter>
            Give a regular expression to include only classes whose fully
            qualified names match.

        -t <tagsFilter>, --filter-tags <tagsFilter>
            Give a tag to include in the test run. This option can be repeated.

        -x, --enable-exit-code
            Exit process with number of failing tests as exit code

        --
            This option can be used to separate command-line options from the
            list of argument, (useful when arguments might be mistaken for
            command-line options

        <arguments>
            Test classes, methods or packages to execute. If --all|-a has been
            chosen, arguments can list all classpath roots that should be
            considered for test scanning, or none if the full classpath shall be
            scanned.

Using JUnit4 to Run JUnit5 Tests

The JUnit5 runner lets you run JUnit 5 tests with JUnit 4. This way you can run JUnit 5 tests in IDEs and build tools that only know about JUnit 4. As soon as we add reporting features to JUnit 5 that JUnit 4 does not have, the runner will only be able to support a subset of the JUnit 5 functionality. But for the time being the JUnit5 runner is an easy way to get started.

Setup

You need the following artifacts and their dependencies on the classpath:

  • junit5-api (org.junit.prototype:junit5-api:5.0.0-SNAPSHOT) in test scope: API for writing tests; includes @Test etc.

  • junit4-launcher-runner (org.junit.prototype:junit4-launcher-runner:5.0.0-SNAPSHOT) in test scope: Location of the JUnit5 runner.

  • junit5-engine (org.junit.prototype:junit5-engine:5.0.0-SNAPSHOT) in testRuntime scope: Implementation of the Engine API for JUnit 5.

Single Test Class

One way to use the JUnit5 runner is to annotate a JUnit 5 test class with @RunWith(JUnit5.class) directly. Please note that the tests are annotated with org.junit.gen5.api.Test (JUnit 5), not org.junit.Test (JUnit 4). Moreover, in this case the test class must be public because; otherwise, the IDEs won't recognize it as a test class.

package com.example;

import static org.junit.gen5.api.Assertions.fail;

import org.junit.gen5.api.Test;
import org.junit.gen5.junit4runner.JUnit5;
import org.junit.runner.RunWith;

@RunWith(JUnit5.class)
public class AJUnit5TestCaseRunWithJUnit4 {

	@Test
	void aSucceedingTest() {
		/* no-op */
	}

	@Test
	void aFailingTest() {
		fail("Failing for failing's sake.");
	}

}

Multiple Tests

If you have multiple JUnit 5 tests you can create a test suite.

package com.example;

import org.junit.gen5.junit4runner.JUnit5;
import org.junit.gen5.junit4runner.JUnit5.Packages;
import org.junit.runner.RunWith;

@RunWith(JUnit5.class)
@Packages("com.example")
public class JUnit4SamplesSuite {
}

This suite will discover and run all tests in the com.example package and its subpackages.

There are more options to discover and filter tests besides @Packages. Please have a look at the Javadoc or the code.

Clone this wiki locally