1. Overview
The goal of this document is to provide comprehensive reference documentation for programmers writing tests, extension authors, and engine authors as well as build tool and IDE vendors.
This document is also available as a PDF download.
1.1. What is JUnit 5?
Unlike previous versions of JUnit, JUnit 5 is composed of several different modules from three different sub-projects.
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage
The JUnit Platform serves as a foundation for launching testing
frameworks on the JVM. It also defines the TestEngine API for developing a testing
framework that runs on the platform. Furthermore, the platform provides a
Console Launcher to launch the platform from the
command line and build plugins for Gradle and
Maven as well as a
JUnit 4 based Runner for running any TestEngine
on the platform.
JUnit Jupiter is the combination of the new programming model and
extension model for writing tests and extensions in JUnit 5. The Jupiter
sub-project provides a TestEngine for running Jupiter based tests on the platform.
JUnit Vintage provides a TestEngine for running JUnit 3 and JUnit 4 based tests on
the platform.
1.2. Supported Java Versions
JUnit 5 requires Java 8 (or higher) at runtime. However, you can still test code that has been compiled with previous versions of the JDK.
1.3. Getting Help
Ask JUnit 5 related questions on Stack Overflow or chat with us on Gitter.
2. Installation
Artifacts for final releases and milestones are deployed to Maven Central.
Snapshot artifacts are deployed to Sonatype’s snapshots repository under /org/junit.
2.1. Dependency Metadata
2.1.1. JUnit Platform
-
Group ID:
org.junit.platform -
Version:
1.0.0 -
Artifact IDs:
junit-platform-commons-
Internal common library/utilities of JUnit. These utilities are intended solely for usage within the JUnit framework itself. Any usage by external parties is not supported. Use at your own risk!
junit-platform-console-
Support for discovering and executing tests on the JUnit Platform from the console. See Console Launcher for details.
junit-platform-console-standalone-
An executable JAR with all dependencies included is provided at Maven Central under the junit-platform-console-standalone directory. See Console Launcher for details.
junit-platform-engine-
Public API for test engines. See Plugging in Your Own Test Engine for details.
junit-platform-gradle-plugin-
Support for discovering and executing tests on the JUnit Platform using Gradle.
junit-platform-launcher-
Public API for configuring and launching test plans — typically used by IDEs and build tools. See JUnit Platform Launcher API for details.
junit-platform-runner-
Runner for executing tests and test suites on the JUnit Platform in a JUnit 4 environment. See Using JUnit 4 to Run the JUnit Platform for details.
junit-platform-suite-api-
Annotations for configuring test suites on the JUnit Platform. Supported by the JUnitPlatform runner and possibly by third-party
TestEngineimplementations. junit-platform-surefire-provider-
Support for discovering and executing tests on the JUnit Platform using Maven Surefire.
2.1.2. JUnit Jupiter
-
Group ID:
org.junit.jupiter -
Version:
5.0.0 -
Artifact IDs:
junit-jupiter-api-
JUnit Jupiter API for writing tests and extensions.
junit-jupiter-engine-
JUnit Jupiter test engine implementation, only required at runtime.
junit-jupiter-params-
Support for parameterized tests in JUnit Jupiter.
junit-jupiter-migrationsupport-
Migration support from JUnit 4 to JUnit Jupiter, only required for running selected JUnit 4 rules.
2.3. JUnit Jupiter Sample Projects
The junit5-samples repository hosts a collection of sample
projects based on JUnit Jupiter and JUnit Vintage. You’ll find the respective
build.gradle and pom.xml in the projects below.
-
For Gradle, check out the
junit5-gradle-consumerproject. -
For Maven, check out the
junit5-maven-consumerproject.
3. Writing Tests
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
class FirstJUnit5Tests {
@Test
void myFirstTest() {
assertEquals(2, 1 + 1);
}
}3.1. Annotations
JUnit Jupiter supports the following annotations for configuring tests and extending the framework.
All core annotations are located in the org.junit.jupiter.api package in the junit-jupiter-api
module.
| Annotation | Description |
|---|---|
|
Denotes that a method is a test method. Unlike JUnit 4’s |
|
Denotes that a method is a parameterized test. Such methods are inherited unless they are overridden. |
|
Denotes that a method is a test template for a repeated test. Such methods are inherited unless they are overridden. |
|
Denotes that a method is a test factory for dynamic tests. Such methods are inherited unless they are overridden. |
|
Used to configure the test instance lifecycle for the annotated test class. Such annotations are inherited. |
|
Denotes that a method is a template for test cases designed to be invoked multiple times depending on the number of invocation contexts returned by the registered providers. Such methods are inherited unless they are overridden. |
|
Declares a custom display name for the test class or test method. Such annotations are not inherited. |
|
Denotes that the annotated method should be executed before each |
|
Denotes that the annotated method should be executed after each |
|
Denotes that the annotated method should be executed before all |
|
Denotes that the annotated method should be executed after all |
|
Denotes that the annotated class is a nested, non-static test class. |
|
Used to declare tags for filtering tests, either at the class or method level; analogous to test groups in TestNG or Categories in JUnit 4. Such annotations are inherited at the class level but not at the method level. |
|
Used to disable a test class or test method; analogous to JUnit 4’s |
|
Used to register custom extensions. Such annotations are inherited. |
Methods annotated with @Test, @TestTemplate, @RepeatedTest, @BeforeAll,
@AfterAll, @BeforeEach, or @AfterEach annotations must not return a value.
| Some annotations may currently be experimental. Consult the table in Experimental APIs for details. |
3.1.1. Meta-Annotations and Composed Annotations
JUnit Jupiter annotations can be used as meta-annotations. That means that you can define your own composed annotation that will automatically inherit the semantics of its meta-annotations.
For example, instead of copying and pasting @Tag("fast") throughout your code base (see
Tagging and Filtering), you can create a custom composed annotation
named @Fast as follows. @Fast can then be used as a drop-in replacement for
@Tag("fast").
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import org.junit.jupiter.api.Tag;
@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Tag("fast")
public @interface Fast {
}3.2. Standard Test Class
import static org.junit.jupiter.api.Assertions.fail;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
class StandardTests {
@BeforeAll
static void initAll() {
}
@BeforeEach
void init() {
}
@Test
void succeedingTest() {
}
@Test
void failingTest() {
fail("a failing test");
}
@Test
@Disabled("for demonstration purposes")
void skippedTest() {
// not executed
}
@AfterEach
void tearDown() {
}
@AfterAll
static void tearDownAll() {
}
}
Neither test classes nor test methods need to be public.
|
3.3. Display Names
Test classes and test methods can declare custom display names — with spaces, special characters, and even emojis — that will be displayed by test runners and test reporting.
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
@DisplayName("A special test case")
class DisplayNameDemo {
@Test
@DisplayName("Custom test name containing spaces")
void testWithDisplayNameContainingSpaces() {
}
@Test
@DisplayName("╯°□°）╯")
void testWithDisplayNameContainingSpecialCharacters() {
}
@Test
@DisplayName("😱")
void testWithDisplayNameContainingEmoji() {
}
}3.4. Assertions
JUnit Jupiter comes with many of the assertion methods that JUnit 4 has and adds a few
that lend themselves well to being used with Java 8 lambdas. All JUnit Jupiter assertions
are static methods in the org.junit.jupiter.Assertions class.
import static java.time.Duration.ofMillis;
import static java.time.Duration.ofMinutes;
import static org.junit.jupiter.api.Assertions.assertAll;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTimeout;
import static org.junit.jupiter.api.Assertions.assertTimeoutPreemptively;
import static org.junit.jupiter.api.Assertions.assertTrue;
import org.junit.jupiter.api.Test;
class AssertionsDemo {
@Test
void standardAssertions() {
assertEquals(2, 2);
assertEquals(4, 4, "The optional assertion message is now the last parameter.");
assertTrue(2 == 2, () -> "Assertion messages can be lazily evaluated -- "
+ "to avoid constructing complex messages unnecessarily.");
}
@Test
void groupedAssertions() {
// In a grouped assertion all assertions are executed, and any
// failures will be reported together.
assertAll("person",
() -> assertEquals("John", person.getFirstName()),
() -> assertEquals("Doe", person.getLastName())
);
}
@Test
void dependentAssertions() {
// Within a code block, if an assertion fails the
// subsequent code in the same block will be skipped.
assertAll("properties",
() -> {
String firstName = person.getFirstName();
assertNotNull(firstName);
// Executed only if the previous assertion is valid.
assertAll("first name",
() -> assertTrue(firstName.startsWith("J")),
() -> assertTrue(firstName.endsWith("n"))
);
},
() -> {
// Grouped assertion, so processed independently
// of results of first name assertions.
String lastName = person.getLastName();
assertNotNull(lastName);
// Executed only if the previous assertion is valid.
assertAll("last name",
() -> assertTrue(lastName.startsWith("D")),
() -> assertTrue(lastName.endsWith("e"))
);
}
);
}
@Test
void exceptionTesting() {
Throwable exception = assertThrows(IllegalArgumentException.class, () -> {
throw new IllegalArgumentException("a message");
});
assertEquals("a message", exception.getMessage());
}
@Test
void timeoutNotExceeded() {
// The following assertion succeeds.
assertTimeout(ofMinutes(2), () -> {
// Perform task that takes less than 2 minutes.
});
}
@Test
void timeoutNotExceededWithResult() {
// The following assertion succeeds, and returns the supplied object.
String actualResult = assertTimeout(ofMinutes(2), () -> {
return "a result";
});
assertEquals("a result", actualResult);
}
@Test
void timeoutNotExceededWithMethod() {
// The following assertion invokes a method reference and returns an object.
String actualGreeting = assertTimeout(ofMinutes(2), AssertionsDemo::greeting);
assertEquals("hello world!", actualGreeting);
}
@Test
void timeoutExceeded() {
// The following assertion fails with an error message similar to:
// execution exceeded timeout of 10 ms by 91 ms
assertTimeout(ofMillis(10), () -> {
// Simulate task that takes more than 10 ms.
Thread.sleep(100);
});
}
@Test
void timeoutExceededWithPreemptiveTermination() {
// The following assertion fails with an error message similar to:
// execution timed out after 10 ms
assertTimeoutPreemptively(ofMillis(10), () -> {
// Simulate task that takes more than 10 ms.
Thread.sleep(100);
});
}
private static String greeting() {
return "hello world!";
}
}3.4.1. Third-party Assertion Libraries
Even though the assertion facilities provided by JUnit Jupiter are sufficient for many testing scenarios, there are times when more power and additional functionality such as matchers are desired or required. In such cases, the JUnit team recommends the use of third-party assertion libraries such as AssertJ, Hamcrest, Truth, etc. Developers are therefore free to use the assertion library of their choice.
For example, the combination of matchers and a fluent API can be used to make
assertions more descriptive and readable. However, JUnit Jupiter’s org.junit.jupiter.Assertions class
does not provide an
assertThat()
method like the one found in JUnit 4’s org.junit.Assert class which accepts a Hamcrest
Matcher. Instead,
developers are encouraged to use the built-in support for matchers provided by third-party
assertion libraries.
The following example demonstrates how to use the assertThat() support from Hamcrest in
a JUnit Jupiter test. As long as the Hamcrest library has been added to the classpath,
you can statically import methods such as assertThat(), is(), and equalTo() and
then use them in tests like in the assertWithHamcrestMatcher() method below.
import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.MatcherAssert.assertThat;
import org.junit.jupiter.api.Test;
class HamcrestAssertionDemo {
@Test
void assertWithHamcrestMatcher() {
assertThat(2 + 1, is(equalTo(3)));
}
}Naturally, legacy tests based on the JUnit 4 programming model can continue using
org.junit.Assert#assertThat.
3.5. Assumptions
JUnit Jupiter comes with a subset of the assumption methods that JUnit 4 provides and
adds a few that lend themselves well to being used with Java 8 lambdas. All JUnit Jupiter
assumptions are static methods in the org.junit.jupiter.Assumptions class.
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assumptions.assumeTrue;
import static org.junit.jupiter.api.Assumptions.assumingThat;
import org.junit.jupiter.api.Test;
class AssumptionsDemo {
@Test
void testOnlyOnCiServer() {
assumeTrue("CI".equals(System.getenv("ENV")));
// remainder of test
}
@Test
void testOnlyOnDeveloperWorkstation() {
assumeTrue("DEV".equals(System.getenv("ENV")),
() -> "Aborting test: not on developer workstation");
// remainder of test
}
@Test
void testInAllEnvironments() {
assumingThat("CI".equals(System.getenv("ENV")),
() -> {
// perform these assertions only on the CI server
assertEquals(2, 2);
});
// perform these assertions in all environments
assertEquals("a string", "a string");
}
}3.6. Disabling Tests
Here’s a disabled test case.
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
@Disabled
class DisabledClassDemo {
@Test
void testWillBeSkipped() {
}
}And here’s a test case with a disabled test method.
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
class DisabledTestsDemo {
@Disabled
@Test
void testWillBeSkipped() {
}
@Test
void testWillBeExecuted() {
}
}3.7. Tagging and Filtering
Test classes and methods can be tagged. Those tags can later be used to filter test discovery and execution.
3.7.1. Syntax Rules for Tags
-
A tag must not be
nullor blank. -
A trimmed tag must not contain whitespace.
-
A trimmed tag must not contain ISO control characters.
-
A trimmed tag must not contain any of the following reserved characters.
-
,,(,),&,|,!
-
| In the above context, "trimmed" means that leading and trailing whitespace characters have been removed. |
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
@Tag("fast")
@Tag("model")
class TaggingDemo {
@Test
@Tag("taxes")
void testingTaxCalculation() {
}
}3.8. Test Instance Lifecycle
In order to allow individual test methods to be executed in isolation and to avoid unexpected side effects due to mutable test instance state, JUnit creates a new instance of each test class before executing each test method (see note below for what qualifies as a test method). This "per-method" test instance lifecycle is the default behavior in JUnit Jupiter and is analogous to all previous versions of JUnit.
If you would prefer that JUnit Jupiter execute all test methods on the same test
instance, simply annotate your test class with @TestInstance(Lifecycle.PER_CLASS). When
using this mode, a new test instance will be created once per test class. Thus, if your
test methods rely on state stored in instance variables, you may need to reset that state
in @BeforeEach or @AfterEach methods.
The "per-class" mode has some additional benefits over the default "per-method" mode.
Specifically, with the "per-class" mode it becomes possible to declare @BeforeAll and
@AfterAll on non-static methods as well as on interface default methods. The
"per-class" mode therefore also makes it possible to use @BeforeAll and @AfterAll
methods in @Nested test classes.
If you are authoring tests using the Kotlin programming language, you may also find it
easier to implement @BeforeAll and @AfterAll methods by switching to the "per-class"
test instance lifecycle mode.
In the context of test instance lifecycle a test method is any method annotated
with @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, or @TestTemplate.
|
3.8.1. Changing the Default Test Instance Lifecycle
If a test class or test interface is not annotated with @TestInstance, JUnit Jupiter
will use a default lifecycle mode. The standard default mode is PER_METHOD;
however, it is possible to change the default for the execution of an entire test plan.
To change the default test instance lifecycle mode, simply set the
junit.jupiter.testinstance.lifecycle.default configuration parameter to the name of
an enum constant defined in TestInstance.Lifecycle, ignoring case. This can be supplied
as a JVM system property, as a configuration parameter in the
LauncherDiscoveryRequest that is passed to the Launcher, or via the JUnit Platform
configuration file (see Configuration Parameters for details).
For example, to set the default test instance lifecycle mode to Lifecycle.PER_CLASS,
you can start your JVM with the following system property.
-Djunit.jupiter.testinstance.lifecycle.default=per_class
Note, however, that setting the default test instance lifecycle mode via the JUnit Platform configuration file is a more robust solution since the configuration file can be checked into a version control system along with your project and can therefore be used within IDEs and your build software.
To set the default test instance lifecycle mode to Lifecycle.PER_CLASS via the JUnit
Platform configuration file, create a file named junit-platform.properties in the root
of the class path (e.g., src/test/resources) with the following content.
junit.jupiter.testinstance.lifecycle.default = per_class
| Changing the default test instance lifecycle mode can lead to unpredictable results and fragile builds if not applied consistently. For example, if the build configures "per-class" semantics as the default but tests in the IDE are executed using "per-method" semantics, that can make it difficult to debug errors that occur on the build server. It is therefore recommended to change the default in the JUnit Platform configuration file instead of via a JVM system property. |
3.9. Nested Tests
Nested tests give the test writer more capabilities to express the relationship among several group of tests. Here’s an elaborate example.
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.EmptyStackException;
import java.util.Stack;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
@DisplayName("A stack")
class TestingAStackDemo {
Stack<Object> stack;
@Test
@DisplayName("is instantiated with new Stack()")
void isInstantiatedWithNew() {
new Stack<>();
}
@Nested
@DisplayName("when new")
class WhenNew {
@BeforeEach
void createNewStack() {
stack = new Stack<>();
}
@Test
@DisplayName("is empty")
void isEmpty() {
assertTrue(stack.isEmpty());
}
@Test
@DisplayName("throws EmptyStackException when popped")
void throwsExceptionWhenPopped() {
assertThrows(EmptyStackException.class, () -> stack.pop());
}
@Test
@DisplayName("throws EmptyStackException when peeked")
void throwsExceptionWhenPeeked() {
assertThrows(EmptyStackException.class, () -> stack.peek());
}
@Nested
@DisplayName("after pushing an element")
class AfterPushing {
String anElement = "an element";
@BeforeEach
void pushAnElement() {
stack.push(anElement);
}
@Test
@DisplayName("it is no longer empty")
void isNotEmpty() {
assertFalse(stack.isEmpty());
}
@Test
@DisplayName("returns the element when popped and is empty")
void returnElementWhenPopped() {
assertEquals(anElement, stack.pop());
assertTrue(stack.isEmpty());
}
@Test
@DisplayName("returns the element when peeked but remains not empty")
void returnElementWhenPeeked() {
assertEquals(anElement, stack.peek());
assertFalse(stack.isEmpty());
}
}
}
}
Only non-static nested classes (i.e. inner classes) can serve as @Nested test
classes. Nesting can be arbitrarily deep, and those inner classes are considered to be
full members of the test class family with one exception: @BeforeAll and @AfterAll
methods do not work by default. The reason is that Java does not allow static members
in inner classes. However, this restriction can be circumvented by annotating a @Nested
test class with @TestInstance(Lifecycle.PER_CLASS) (see
Test Instance Lifecycle).
|
3.10. Dependency Injection for Constructors and Methods
In all prior JUnit versions, test constructors or methods were not allowed to have
parameters (at least not with the standard Runner implementations). As one of the major
changes in JUnit Jupiter, both test constructors and methods are now permitted to have
parameters. This allows for greater flexibility and enables Dependency Injection for
constructors and methods.
ParameterResolver defines the API for test extensions that wish to dynamically
resolve parameters at runtime. If a test constructor or a @Test, @TestFactory,
@BeforeEach, @AfterEach, @BeforeAll, or @AfterAll method accepts a parameter, the
parameter must be resolved at runtime by a registered ParameterResolver.
There are currently three built-in resolvers that are registered automatically.
-
TestInfoParameterResolver: if a method parameter is of typeTestInfo, theTestInfoParameterResolverwill supply an instance ofTestInfocorresponding to the current test as the value for the parameter. TheTestInfocan then be used to retrieve information about the current test such as the test’s display name, the test class, the test method, or associated tags. The display name is either a technical name, such as the name of the test class or test method, or a custom name configured via@DisplayName.TestInfoacts as a drop-in replacement for theTestNamerule from JUnit 4. The following demonstrates how to haveTestInfoinjected into a test constructor,@BeforeEachmethod, and@Testmethod.
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInfo;
@DisplayName("TestInfo Demo")
class TestInfoDemo {
TestInfoDemo(TestInfo testInfo) {
assertEquals("TestInfo Demo", testInfo.getDisplayName());
}
@BeforeEach
void init(TestInfo testInfo) {
String displayName = testInfo.getDisplayName();
assertTrue(displayName.equals("TEST 1") || displayName.equals("test2()"));
}
@Test
@DisplayName("TEST 1")
@Tag("my-tag")
void test1(TestInfo testInfo) {
assertEquals("TEST 1", testInfo.getDisplayName());
assertTrue(testInfo.getTags().contains("my-tag"));
}
@Test
void test2() {
}
}-
RepetitionInfoParameterResolver: if a method parameter in a@RepeatedTest,@BeforeEach, or@AfterEachmethod is of typeRepetitionInfo, theRepetitionInfoParameterResolverwill supply an instance ofRepetitionInfo.RepetitionInfocan then be used to retrieve information about the current repetition and the total number of repetitions for the corresponding@RepeatedTest. Note, however, thatRepetitionInfoParameterResolveris not registered outside the context of a@RepeatedTest. See Repeated Test Examples. -
TestReporterParameterResolver: if a method parameter is of typeTestReporter, theTestReporterParameterResolverwill supply an instance ofTestReporter. TheTestReportercan be used to publish additional data about the current test run. The data can be consumed throughTestExecutionListener.reportingEntryPublished()and thus be viewed by IDEs or included in reports.In JUnit Jupiter you should use
TestReporterwhere you used to print information tostdoutorstderrin JUnit 4. Using@RunWith(JUnitPlatform.class)will even output all reported entries tostdout.
import java.util.HashMap;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestReporter;
class TestReporterDemo {
@Test
void reportSingleValue(TestReporter testReporter) {
testReporter.publishEntry("a key", "a value");
}
@Test
void reportSeveralValues(TestReporter testReporter) {
HashMap<String, String> values = new HashMap<>();
values.put("user name", "dk38");
values.put("award year", "1974");
testReporter.publishEntry(values);
}
}
Other parameter resolvers must be explicitly enabled by registering appropriate
extensions via @ExtendWith.
|
Check out the MockitoExtension for an example of a custom ParameterResolver.
While not intended to be production-ready, it demonstrates the simplicity and
expressiveness of both the extension model and the parameter resolution process.
MyMockitoTest demonstrates how to inject Mockito mocks into @BeforeEach and @Test
methods.
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.Mockito.when;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import com.example.Person;
import com.example.mockito.MockitoExtension;
@ExtendWith(MockitoExtension.class)
class MyMockitoTest {
@BeforeEach
void init(@Mock Person person) {
when(person.getName()).thenReturn("Dilbert");
}
@Test
void simpleTestWithInjectedMock(@Mock Person person) {
assertEquals("Dilbert", person.getName());
}
}3.11. Test Interfaces and Default Methods
JUnit Jupiter allows @Test, @RepeatedTest, @ParameterizedTest, @TestFactory,
@TestTemplate, @BeforeEach, and @AfterEach to be declared on interface default
methods. @BeforeAll and @AfterAll can either be declared on static methods in a
test interface or on interface default methods if the test interface or test class is
annotated with @TestInstance(Lifecycle.PER_CLASS) (see
Test Instance Lifecycle). Here are some examples.
@TestInstance(Lifecycle.PER_CLASS)
interface TestLifecycleLogger {
static final Logger LOG = Logger.getLogger(TestLifecycleLogger.class.getName());
@BeforeAll
default void beforeAllTests() {
LOG.info("Before all tests");
}
@AfterAll
default void afterAllTests() {
LOG.info("After all tests");
}
@BeforeEach
default void beforeEachTest(TestInfo testInfo) {
LOG.info(() -> String.format("About to execute [%s]",
testInfo.getDisplayName()));
}
@AfterEach
default void afterEachTest(TestInfo testInfo) {
LOG.info(() -> String.format("Finished executing [%s]",
testInfo.getDisplayName()));
}
}interface TestInterfaceDynamicTestsDemo {
@TestFactory
default Collection<DynamicTest> dynamicTestsFromCollection() {
return Arrays.asList(
dynamicTest("1st dynamic test in test interface", () -> assertTrue(true)),
dynamicTest("2nd dynamic test in test interface", () -> assertEquals(4, 2 * 2))
);
}
}@ExtendWith and @Tag can be declared on a test interface so that classes that
implement the interface automatically inherit its tags and extensions. See
Before and After Test Execution Callbacks for the source code of the
TimingExtension.
@Tag("timed")
@ExtendWith(TimingExtension.class)
interface TimeExecutionLogger {
}In your test class you can then implement these test interfaces to have them applied.
class TestInterfaceDemo implements TestLifecycleLogger,
TimeExecutionLogger, TestInterfaceDynamicTestsDemo {
@Test
void isEqualValue() {
assertEquals(1, 1, "is always equal");
}
}Running the TestInterfaceDemo results in output similar to the following:
:junitPlatformTest INFO example.TestLifecycleLogger - Before all tests INFO example.TestLifecycleLogger - About to execute [dynamicTestsFromCollection()] INFO example.TimingExtension - Method [dynamicTestsFromCollection] took 13 ms. INFO example.TestLifecycleLogger - Finished executing [dynamicTestsFromCollection()] INFO example.TestLifecycleLogger - About to execute [isEqualValue()] INFO example.TimingExtension - Method [isEqualValue] took 1 ms. INFO example.TestLifecycleLogger - Finished executing [isEqualValue()] INFO example.TestLifecycleLogger - After all tests Test run finished after 190 ms [ 3 containers found ] [ 0 containers skipped ] [ 3 containers started ] [ 0 containers aborted ] [ 3 containers successful ] [ 0 containers failed ] [ 3 tests found ] [ 0 tests skipped ] [ 3 tests started ] [ 0 tests aborted ] [ 3 tests successful ] [ 0 tests failed ] BUILD SUCCESSFUL
Another possible application of this feature is to write tests for interface contracts.
For example, you can write tests for how implementations of Object.equals or
Comparable.compareTo should behave as follows.
public interface Testable<T> {
T createValue();
}public interface EqualsContract<T> extends Testable<T> {
T createNotEqualValue();
@Test
default void valueEqualsItself() {
T value = createValue();
assertEquals(value, value);
}
@Test
default void valueDoesNotEqualNull() {
T value = createValue();
assertFalse(value.equals(null));
}
@Test
default void valueDoesNotEqualDifferentValue() {
T value = createValue();
T differentValue = createNotEqualValue();
assertNotEquals(value, differentValue);
assertNotEquals(differentValue, value);
}
}public interface ComparableContract<T extends Comparable<T>> extends Testable<T> {
T createSmallerValue();
@Test
default void returnsZeroWhenComparedToItself() {
T value = createValue();
assertEquals(0, value.compareTo(value));
}
@Test
default void returnsPositiveNumberComparedToSmallerValue() {
T value = createValue();
T smallerValue = createSmallerValue();
assertTrue(value.compareTo(smallerValue) > 0);
}
@Test
default void returnsNegativeNumberComparedToSmallerValue() {
T value = createValue();
T smallerValue = createSmallerValue();
assertTrue(smallerValue.compareTo(value) < 0);
}
}In your test class you can then implement both contract interfaces thereby inheriting the corresponding tests. Of course you’ll have to implement the abstract methods.
class StringTests implements ComparableContract<String>, EqualsContract<String> {
@Override
public String createValue() {
return "foo";
}
@Override
public String createSmallerValue() {
return "bar"; // 'b' < 'f' in "foo"
}
@Override
public String createNotEqualValue() {
return "baz";
}
}| The above tests are merely meant as examples and therefore not complete. |
3.12. Repeated Tests
JUnit Jupiter provides the ability to repeat a test a specified number of times simply by
annotating a method with @RepeatedTest and specifying the total number of repetitions
desired. Each invocation of a repeated test behaves like the execution of a regular
@Test method with full support for the same lifecycle callbacks and extensions.
The following example demonstrates how to declare a test named repeatedTest() that
will be automatically repeated 10 times.
@RepeatedTest(10)
void repeatedTest() {
// ...
}In addition to specifying the number of repetitions, a custom display name can be
configured for each repetition via the name attribute of the @RepeatedTest
annotation. Furthermore, the display name can be a pattern composed of a combination of
static text and dynamic placeholders. The following placeholders are currently supported.
-
{displayName}: display name of the@RepeatedTestmethod -
{currentRepetition}: the current repetition count -
{totalRepetitions}: the total number of repetitions
The default display name for a given repetition is generated based on the following
pattern: "repetition {currentRepetition} of {totalRepetitions}". Thus, the display
names for individual repetitions of the previous repeatedTest() example would be:
repetition 1 of 10, repetition 2 of 10, etc. If you would like the display name of
the @RepeatedTest method included in the name of each repetition, you can define your
own custom pattern or use the predefined RepeatedTest.LONG_DISPLAY_NAME pattern. The
latter is equal to "{displayName} :: repetition {currentRepetition} of
{totalRepetitions}" which results in display names for individual repetitions like
repeatedTest() :: repetition 1 of 10, repeatedTest() :: repetition 2 of 10, etc.
In order to retrieve information about the current repetition and the total number of
repetitions programmatically, a developer can choose to have an instance of
RepetitionInfo injected into a @RepeatedTest, @BeforeEach, or @AfterEach method.
3.12.1. Repeated Test Examples
The RepeatedTestsDemo class at the end of this section demonstrates several examples of
repeated tests.
The repeatedTest() method is identical to example from the previous section; whereas,
repeatedTestWithRepetitionInfo() demonstrates how to have an instance of
RepetitionInfo injected into a test to access the total number of repetitions for the
current repeated test.
The next two methods demonstrate how to include a custom @DisplayName for the
@RepeatedTest method in the display name of each repetition. customDisplayName()
combines a custom display name with a custom pattern and then uses TestInfo to verify
the format of the generated display name. Repeat! is the {displayName} which comes
from the @DisplayName declaration, and 1/1 comes from
{currentRepetition}/{totalRepetitions}. In contrast,
customDisplayNameWithLongPattern() uses the aforementioned predefined
RepeatedTest.LONG_DISPLAY_NAME pattern.
repeatedTestInGerman() demonstrates the ability to translate display names of repeated
tests into foreign languages — in this case German, resulting in names for individual
repetitions such as: Wiederholung 1 von 5, Wiederholung 2 von 5, etc.
Since the beforeEach() method is annotated with @BeforeEach it will get executed
before each repetition of each repeated test. By having the TestInfo and
RepetitionInfo injected into the method, we see that it’s possible to obtain
information about the currently executing repeated test. Executing RepeatedTestsDemo
with the INFO log level enabled results in the following output.
INFO: About to execute repetition 1 of 10 for repeatedTest INFO: About to execute repetition 2 of 10 for repeatedTest INFO: About to execute repetition 3 of 10 for repeatedTest INFO: About to execute repetition 4 of 10 for repeatedTest INFO: About to execute repetition 5 of 10 for repeatedTest INFO: About to execute repetition 6 of 10 for repeatedTest INFO: About to execute repetition 7 of 10 for repeatedTest INFO: About to execute repetition 8 of 10 for repeatedTest INFO: About to execute repetition 9 of 10 for repeatedTest INFO: About to execute repetition 10 of 10 for repeatedTest INFO: About to execute repetition 1 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 2 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 3 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 4 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 5 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 1 of 1 for customDisplayName INFO: About to execute repetition 1 of 1 for customDisplayNameWithLongPattern INFO: About to execute repetition 1 of 5 for repeatedTestInGerman INFO: About to execute repetition 2 of 5 for repeatedTestInGerman INFO: About to execute repetition 3 of 5 for repeatedTestInGerman INFO: About to execute repetition 4 of 5 for repeatedTestInGerman INFO: About to execute repetition 5 of 5 for repeatedTestInGerman
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.util.logging.Logger;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.RepeatedTest;
import org.junit.jupiter.api.RepetitionInfo;
import org.junit.jupiter.api.TestInfo;
class RepeatedTestsDemo {
private Logger logger = // ...
@BeforeEach
void beforeEach(TestInfo testInfo, RepetitionInfo repetitionInfo) {
int currentRepetition = repetitionInfo.getCurrentRepetition();
int totalRepetitions = repetitionInfo.getTotalRepetitions();
String methodName = testInfo.getTestMethod().get().getName();
logger.info(String.format("About to execute repetition %d of %d for %s", //
currentRepetition, totalRepetitions, methodName));
}
@RepeatedTest(10)
void repeatedTest() {
// ...
}
@RepeatedTest(5)
void repeatedTestWithRepetitionInfo(RepetitionInfo repetitionInfo) {
assertEquals(5, repetitionInfo.getTotalRepetitions());
}
@RepeatedTest(value = 1, name = "{displayName} {currentRepetition}/{totalRepetitions}")
@DisplayName("Repeat!")
void customDisplayName(TestInfo testInfo) {
assertEquals(testInfo.getDisplayName(), "Repeat! 1/1");
}
@RepeatedTest(value = 1, name = RepeatedTest.LONG_DISPLAY_NAME)
@DisplayName("Details...")
void customDisplayNameWithLongPattern(TestInfo testInfo) {
assertEquals(testInfo.getDisplayName(), "Details... :: repetition 1 of 1");
}
@RepeatedTest(value = 5, name = "Wiederholung {currentRepetition} von {totalRepetitions}")
void repeatedTestInGerman() {
// ...
}
}When using the ConsoleLauncher or the junitPlatformTest Gradle plugin with the
unicode theme enabled, execution of RepeatedTestsDemo results in the following output
to the console.
├─ RepeatedTestsDemo ✔ │ ├─ repeatedTest() ✔ │ │ ├─ repetition 1 of 10 ✔ │ │ ├─ repetition 2 of 10 ✔ │ │ ├─ repetition 3 of 10 ✔ │ │ ├─ repetition 4 of 10 ✔ │ │ ├─ repetition 5 of 10 ✔ │ │ ├─ repetition 6 of 10 ✔ │ │ ├─ repetition 7 of 10 ✔ │ │ ├─ repetition 8 of 10 ✔ │ │ ├─ repetition 9 of 10 ✔ │ │ └─ repetition 10 of 10 ✔ │ ├─ repeatedTestWithRepetitionInfo(RepetitionInfo) ✔ │ │ ├─ repetition 1 of 5 ✔ │ │ ├─ repetition 2 of 5 ✔ │ │ ├─ repetition 3 of 5 ✔ │ │ ├─ repetition 4 of 5 ✔ │ │ └─ repetition 5 of 5 ✔ │ ├─ Repeat! ✔ │ │ └─ Repeat! 1/1 ✔ │ ├─ Details... ✔ │ │ └─ Details... :: repetition 1 of 1 ✔ │ └─ repeatedTestInGerman() ✔ │ ├─ Wiederholung 1 von 5 ✔ │ ├─ Wiederholung 2 von 5 ✔ │ ├─ Wiederholung 3 von 5 ✔ │ ├─ Wiederholung 4 von 5 ✔ │ └─ Wiederholung 5 von 5 ✔
3.13. Parameterized Tests
Parameterized tests make it possible to run a test multiple times with different
arguments. They are declared just like regular @Test methods but use the
@ParameterizedTest annotation instead. In addition, you must declare at least one
source that will provide the arguments for each invocation.
| Parameterized tests are currently an experimental feature. Consult the table in Experimental APIs for details. |
@ParameterizedTest
@ValueSource(strings = { "Hello", "World" })
void testWithStringParameter(String argument) {
assertNotNull(argument);
}This parameterized test uses the @ValueSource annotation to specify a String array as
the source of arguments. When executing this method, each invocation will be reported
separately. For instance, the ConsoleLauncher will print output similar to the
following.
testWithStringParameter(String) ✔ ├─ [1] Hello ✔ └─ [2] World ✔
3.13.1. Required Setup
In order to use parameterized tests you need to add a dependency on the
junit-jupiter-params artifact. Please refer to Dependency Metadata for details.
3.13.2. Sources of Arguments
Out of the box, JUnit Jupiter provides quite a few source annotations. Each of the
following subsections provides a brief overview and an example for each of them. Please
refer to the JavaDoc in the org.junit.jupiter.params.provider package for additional
information.
@ValueSource
@ValueSource is one of the simplest possible sources. It lets you specify an array of
literals of primitive types (either String, int, long, or double) and can only be
used for providing a single parameter per invocation.
@ParameterizedTest
@ValueSource(ints = { 1, 2, 3 })
void testWithValueSource(int argument) {
assertNotNull(argument);
}@EnumSource
@EnumSource provides a convenient way to use Enum constants. The annotation provides
an optional names parameter that lets you specify which constants shall be used. If
omitted, all constants will be used like in the following example.
@ParameterizedTest
@EnumSource(TimeUnit.class)
void testWithEnumSource(TimeUnit timeUnit) {
assertNotNull(timeUnit);
}@ParameterizedTest
@EnumSource(value = TimeUnit.class, names = { "DAYS", "HOURS" })
void testWithEnumSourceInclude(TimeUnit timeUnit) {
assertTrue(EnumSet.of(TimeUnit.DAYS, TimeUnit.HOURS).contains(timeUnit));
}The @EnumSource annotation also provides an optional mode parameter that enables
fine-grained control over which constants are passed to the test method. For example, you
can exclude names from the enum constant pool or specify regular expressions as in the
following examples.
@ParameterizedTest
@EnumSource(value = TimeUnit.class, mode = EXCLUDE, names = { "DAYS", "HOURS" })
void testWithEnumSourceExclude(TimeUnit timeUnit) {
assertFalse(EnumSet.of(TimeUnit.DAYS, TimeUnit.HOURS).contains(timeUnit));
assertTrue(timeUnit.name().length() > 5);
}@ParameterizedTest
@EnumSource(value = TimeUnit.class, mode = MATCH_ALL, names = "^(M|N).+SECONDS$")
void testWithEnumSourceRegex(TimeUnit timeUnit) {
String name = timeUnit.name();
assertTrue(name.startsWith("M") || name.startsWith("N"));
assertTrue(name.endsWith("SECONDS"));
}@MethodSource
@MethodSource allows you to refer to one or more methods of the test class. Each method
must return a Stream, Iterable, Iterator, or array of arguments. In addition, each
method must not accept any arguments. By default such methods must be static unless the
test class is annotated with @TestInstance(Lifecycle.PER_CLASS).
If you only need a single parameter, you can return instances of the parameter type directly as demonstrated by the following example.
@ParameterizedTest
@MethodSource("stringProvider")
void testWithSimpleMethodSource(String argument) {
assertNotNull(argument);
}
static Stream<String> stringProvider() {
return Stream.of("foo", "bar");
}Streams for primitive types (DoubleStream, IntStream, and LongStream) are also
supported.
@ParameterizedTest
@MethodSource("range")
void testWithRangeMethodSource(int argument) {
assertNotEquals(9, argument);
}
static IntStream range() {
return IntStream.range(0, 20).skip(10);
}In case you need multiple parameters, you need to return an Arguments instances as
shown below. Note that Arguments.of(Object…) is a static factory method defined in
the interface itself.
@ParameterizedTest
@MethodSource("stringAndIntProvider")
void testWithMultiArgMethodSource(String first, int second) {
assertNotNull(first);
assertNotEquals(0, second);
}
static Stream<Arguments> stringAndIntProvider() {
return Stream.of(Arguments.of("foo", 1), Arguments.of("bar", 2));
}@CsvSource
@CsvSource allows you to express argument lists as comma-separated values (i.e.,
String literals).
@ParameterizedTest
@CsvSource({ "foo, 1", "bar, 2", "'baz, qux', 3" })
void testWithCsvSource(String first, int second) {
assertNotNull(first);
assertNotEquals(0, second);
}@CsvSource uses a single quote ' as its quote character. See the 'baz, qux' value
in the example above and in the table below. An empty, quoted value '' results in an
empty String; whereas, an entirely empty value is interpreted as a null reference.
An ArgumentConversionException is raised if the target type of a null reference is a
primitive type.
| Example Input | Resulting Argument List |
|---|---|
|
|
|
|
|
|
|
|
@CsvFileSource
@CsvFileSource lets you use CSV files from the classpath. Each line from a CSV file
results in one invocation of the parameterized test.
@ParameterizedTest
@CsvFileSource(resources = "/two-column.csv")
void testWithCsvFileSource(String first, int second) {
assertNotNull(first);
assertNotEquals(0, second);
}foo, 1
bar, 2
"baz, qux", 3
In contrast to the syntax used in @CsvSource, @CsvFileSource uses a double
quote " as the quote character. See the "baz, qux" value in the example above. An
empty, quoted value "" results in an empty String; whereas, an entirely empty value
is interpreted as a null reference. An ArgumentConversionException is raised if the
target type of a null reference is a primitive type.
|
@ArgumentsSource
@ArgumentsSource can be used to specify a custom, reusable ArgumentsProvider.
@ParameterizedTest
@ArgumentsSource(MyArgumentsProvider.class)
void testWithArgumentsSource(String argument) {
assertNotNull(argument);
}
static class MyArgumentsProvider implements ArgumentsProvider {
@Override
public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
return Stream.of("foo", "bar").map(Arguments::of);
}
}3.13.3. Argument Conversion
Implicit Conversion
To support use cases like @CsvSource, JUnit Jupiter provides a number of built-in
implicit type converters. The conversion process depends on the declared type of each
method parameter.
For example, if a @ParameterizedTest declares a parameter of type TimeUnit and the
actual type supplied by the declared source is a String, the string will automatically
be converted into the corresponding TimeUnit enum constant.
@ParameterizedTest
@ValueSource(strings = "SECONDS")
void testWithImplicitArgumentConversion(TimeUnit argument) {
assertNotNull(argument.name());
}String instances are currently implicitly converted to the following target types.
| Target Type | Example |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Explicit Conversion
Instead of using implicit argument conversion you may explicitly specify an
ArgumentConverter to use for a certain parameter using the @ConvertWith annotation
like in the following example.
@ParameterizedTest
@EnumSource(TimeUnit.class)
void testWithExplicitArgumentConversion(@ConvertWith(ToStringArgumentConverter.class) String argument) {
assertNotNull(TimeUnit.valueOf(argument));
}
static class ToStringArgumentConverter extends SimpleArgumentConverter {
@Override
protected Object convert(Object source, Class<?> targetType) {
assertEquals(String.class, targetType, "Can only convert to String");
return String.valueOf(source);
}
}Explicit argument converters are meant to be implemented by test authors. Thus,
junit-jupiter-params only provides a single explicit argument converter that may also
serve as a reference implementation: JavaTimeArgumentConverter. It is used via the
composed annotation JavaTimeConversionPattern.
@ParameterizedTest
@ValueSource(strings = { "01.01.2017", "31.12.2017" })
void testWithExplicitJavaTimeConverter(@JavaTimeConversionPattern("dd.MM.yyyy") LocalDate argument) {
assertEquals(2017, argument.getYear());
}3.13.4. Customizing Display Names
By default, the display name of a parameterized test invocation contains the invocation
index and the String representation of all arguments for that specific invocation.
However, you can customize invocation display names via the name attribute of the
@ParameterizedTest annotation like in the following example.
@DisplayName("Display name of container")
@ParameterizedTest(name = "{index} ==> first=''{0}'', second={1}")
@CsvSource({ "foo, 1", "bar, 2", "'baz, qux', 3" })
void testWithCustomDisplayNames(String first, int second) {
}When executing the above method using the ConsoleLauncher you will see output similar to the following.
Display name of container ✔ ├─ 1 ==> first='foo', second=1 ✔ ├─ 2 ==> first='bar', second=2 ✔ └─ 3 ==> first='baz, qux', second=3 ✔
The following placeholders are supported within custom display names.
| Placeholder | Description |
|---|---|
|
the current invocation index (1-based) |
|
the complete, comma-separated arguments list |
|
an individual argument |
3.13.5. Lifecycle and Interoperability
Each invocation of a parameterized test has the same lifecycle as a regular @Test
method. For example, @BeforeEach methods will be executed before each invocation.
Similar to Dynamic Tests, invocations will appear one by one in the
test tree of an IDE. You may at will mix regular @Test methods and @ParameterizedTest
methods within the same test class.
You may use ParameterResolver extensions with @ParameterizedTest methods. However,
method parameters that are resolved by argument sources need to come first in the
argument list. Since a test class may contain regular tests as well as parameterized
tests with different parameter lists, values from argument sources are not resolved for
lifecycle methods (e.g. @BeforeEach) and test class constructors.
@BeforeEach
void beforeEach(TestInfo testInfo) {
// ...
}
@ParameterizedTest
@ValueSource(strings = "foo")
void testWithRegularParameterResolver(String argument, TestReporter testReporter) {
testReporter.publishEntry("argument", argument);
}
@AfterEach
void afterEach(TestInfo testInfo) {
// ...
}3.14. Test Templates
A @TestTemplate method is not a regular test case but rather a template for test
cases. As such, it is designed to be invoked multiple times depending on the number of
invocation contexts returned by the registered providers. Thus, it must be used in
conjunction with a registered TestTemplateInvocationContextProvider extension. Each
invocation of a test template method behaves like the execution of a regular @Test
method with full support for the same lifecycle callbacks and extensions. Please refer to
Providing Invocation Contexts for Test Templates for usage examples.
3.15. Dynamic Tests
The standard @Test annotation in JUnit Jupiter described in
Annotations is very similar to the @Test annotation in JUnit 4. Both
describe methods that implement test cases. These test cases are static in the sense that
they are fully specified at compile time, and their behavior cannot be changed by
anything happening at runtime. Assumptions provide a basic form of dynamic behavior but
are intentionally rather limited in their expressiveness.
In addition to these standard tests a completely new kind of test programming model has
been introduced in JUnit Jupiter. This new kind of test is a dynamic test which is
generated at runtime by a factory method that is annotated with @TestFactory.
In contrast to @Test methods, a @TestFactory method is not itself a test case but
rather a factory for test cases. Thus, a dynamic test is the product of a factory.
Technically speaking, a @TestFactory method must return a Stream, Collection,
Iterable, or Iterator of DynamicNode instances. Instantiable subclasses of
DynamicNode are DynamicContainer and DynamicTest. DynamicContainer instances are
composed of a display name and a list of dynamic child nodes, enabling the creation of
arbitrarily nested hierarchies of dynamic nodes. DynamicTest instances will then be
executed lazily, enabling dynamic and even non-deterministic generation of test cases.
Any Stream returned by a @TestFactory will be properly closed by calling
stream.close(), making it safe to use a resource such as Files.lines().
As with @Test methods, @TestFactory methods must not be private or static and may
optionally declare parameters to be resolved by ParameterResolvers.
A DynamicTest is a test case generated at runtime. It is composed of a display name
and an Executable. Executable is a @FunctionalInterface which means that the
implementations of dynamic tests can be provided as lambda expressions or method
references.
|
Dynamic Test Lifecycle
The execution lifecycle of a dynamic test is quite different than it is for a
standard @Test case. Specifically, there are no lifecycle callbacks for individual
dynamic tests. This means that @BeforeEach and @AfterEach methods and their
corresponding extension callbacks are executed for the @TestFactory method but not for
each dynamic test. In other words, if you access fields from the test instance within a
lambda expression for a dynamic test, those fields will not be reset by callback methods
or extensions between the execution of individual dynamic tests generated by the same
@TestFactory method.
|
As of JUnit Jupiter 5.0.0, dynamic tests must always be created by factory methods; however, this might be complemented by a registration facility in a later release.
| Dynamic tests are currently an experimental feature. Consult the table in Experimental APIs for details. |
3.15.1. Dynamic Test Examples
The following DynamicTestsDemo class demonstrates several examples of test factories
and dynamic tests.
The first method returns an invalid return type. Since an invalid return type cannot be
detected at compile time, a JUnitException is thrown when it is detected at runtime.
The next five methods are very simple examples that demonstrate the generation of a
Collection, Iterable, Iterator, or Stream of DynamicTest instances. Most of
these examples do not really exhibit dynamic behavior but merely demonstrate the
supported return types in principle. However, dynamicTestsFromStream() and
dynamicTestsFromIntStream() demonstrate how easy it is to generate dynamic tests for a
given set of strings or a range of input numbers.
The next method is truly dynamic in nature. generateRandomNumberOfTests() implements an
Iterator that generates random numbers, a display name generator, and a test executor
and then provides all three to DynamicTest.stream(). Although the non-deterministic
behavior of generateRandomNumberOfTests() is of course in conflict with test
repeatability and should thus be used with care, it serves to demonstrate the
expressiveness and power of dynamic tests.
The last method generates a nested hierarchy of dynamic tests utilizing
DynamicContainer.
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.DynamicContainer.dynamicContainer;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;
import java.util.Random;
import java.util.function.Function;
import java.util.stream.IntStream;
import java.util.stream.Stream;
import org.junit.jupiter.api.DynamicNode;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.TestFactory;
import org.junit.jupiter.api.function.ThrowingConsumer;
class DynamicTestsDemo {
// This will result in a JUnitException!
@TestFactory
List<String> dynamicTestsWithInvalidReturnType() {
return Arrays.asList("Hello");
}
@TestFactory
Collection<DynamicTest> dynamicTestsFromCollection() {
return Arrays.asList(
dynamicTest("1st dynamic test", () -> assertTrue(true)),
dynamicTest("2nd dynamic test", () -> assertEquals(4, 2 * 2))
);
}
@TestFactory
Iterable<DynamicTest> dynamicTestsFromIterable() {
return Arrays.asList(
dynamicTest("3rd dynamic test", () -> assertTrue(true)),
dynamicTest("4th dynamic test", () -> assertEquals(4, 2 * 2))
);
}
@TestFactory
Iterator<DynamicTest> dynamicTestsFromIterator() {
return Arrays.asList(
dynamicTest("5th dynamic test", () -> assertTrue(true)),
dynamicTest("6th dynamic test", () -> assertEquals(4, 2 * 2))
).iterator();
}
@TestFactory
Stream<DynamicTest> dynamicTestsFromStream() {
return Stream.of("A", "B", "C")
.map(str -> dynamicTest("test" + str, () -> { /* ... */ }));
}
@TestFactory
Stream<DynamicTest> dynamicTestsFromIntStream() {
// Generates tests for the first 10 even integers.
return IntStream.iterate(0, n -> n + 2).limit(10)
.mapToObj(n -> dynamicTest("test" + n, () -> assertTrue(n % 2 == 0)));
}
@TestFactory
Stream<DynamicTest> generateRandomNumberOfTests() {
// Generates random positive integers between 0 and 100 until
// a number evenly divisible by 7 is encountered.
Iterator<Integer> inputGenerator = new Iterator<Integer>() {
Random random = new Random();
int current;
@Override
public boolean hasNext() {
current = random.nextInt(100);
return current % 7 != 0;
}
@Override
public Integer next() {
return current;
}
};
// Generates display names like: input:5, input:37, input:85, etc.
Function<Integer, String> displayNameGenerator = (input) -> "input:" + input;
// Executes tests based on the current input value.
ThrowingConsumer<Integer> testExecutor = (input) -> assertTrue(input % 7 != 0);
// Returns a stream of dynamic tests.
return DynamicTest.stream(inputGenerator, displayNameGenerator, testExecutor);
}
@TestFactory
Stream<DynamicNode> dynamicTestsWithContainers() {
return Stream.of("A", "B", "C")
.map(input -> dynamicContainer("Container " + input, Stream.of(
dynamicTest("not null", () -> assertNotNull(input)),
dynamicContainer("properties", Stream.of(
dynamicTest("length > 0", () -> assertTrue(input.length() > 0)),
dynamicTest("not empty", () -> assertFalse(input.isEmpty()))
))
)));
}
}4. Running Tests
4.1. IDE Support
4.1.1. IntelliJ IDEA
IntelliJ IDEA supports running tests on the JUnit Platform since version 2016.2. For details please see the post on the IntelliJ IDEA blog.
| IntelliJ IDEA Version | Bundled JUnit 5 Version |
|---|---|
2016.2 |
M2 |
2016.3.1 |
M3 |
2017.1.2 |
M4 |
2017.2.1 |
M5 |
2017.2.3 |
RC2 |
| IntelliJ IDEA bundles a certain version of JUnit 5. That means, if you want to use a newer milestone version of the Jupiter API, executing the tests might not work. This situation will improve once the first GA version of JUnit 5 has been released. In the meantime, please follow the instructions below to use a newer version of JUnit 5 than the one bundled with IntelliJ IDEA. |
In order to use a different JUnit 5 version, you have to manually add the
junit-platform-launcher, junit-jupiter-engine, and junit-vintage-engine JARs to the
classpath.
// Only needed to run tests in an IntelliJ IDEA that bundles an older version
testRuntime("org.junit.platform:junit-platform-launcher:1.0.0")
testRuntime("org.junit.jupiter:junit-jupiter-engine:5.0.0")
testRuntime("org.junit.vintage:junit-vintage-engine:4.12.0")<!-- Only required to run tests in an IntelliJ IDEA that bundles an older version -->
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-launcher</artifactId>
<version>1.0.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.0.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
<version>4.12.0</version>
<scope>test</scope>
</dependency>4.1.2. Eclipse Beta Support
Eclipse 4.7 (Oxygen) has beta support for the JUnit Platform and JUnit Jupiter. For details on how to set it up, please consult the Eclipse JDT UI/JUnit 5 wiki page.
4.1.3. Other IDEs
At the time of this writing, there is no direct support for running tests on the JUnit Platform within IDEs other than with IntelliJ IDEA or the beta support in Eclipse. However, the JUnit team provides two intermediate solutions so that you can go ahead and try out JUnit 5 within your IDE today. You can use the Console Launcher manually or execute tests with a JUnit 4 based Runner.
4.2. Build Support
4.2.1. Gradle
The JUnit team has developed a very basic Gradle plugin that allows you to run any kind
of test that is supported by a TestEngine (e.g., JUnit 3, JUnit 4, JUnit Jupiter,
Specsy, etc.). See build.gradle in the junit5-gradle-consumer project for an
example of the plugin in action.
Enabling the JUnit Gradle Plugin
To use the JUnit Gradle plugin, you first need to make sure that you are running Gradle
2.5 or higher. Once you’ve done that, you can configure build.gradle as follows.
buildscript {
repositories {
mavenCentral()
// The following is only necessary if you want to use SNAPSHOT releases.
// maven { url 'https://oss.sonatype.org/content/repositories/snapshots' }
}
dependencies {
classpath 'org.junit.platform:junit-platform-gradle-plugin:1.0.0'
}
}
apply plugin: 'org.junit.platform.gradle.plugin'Configuring the JUnit Gradle Plugin
Once the JUnit Gradle plugin has been applied, you can configure it as follows.
junitPlatform {
platformVersion 1.0
logManager 'org.apache.logging.log4j.jul.LogManager'
reportsDir file('build/test-results/junit-platform') // this is the default
// enableStandardTestTask true
// selectors (optional)
// filters (optional)
}Setting logManager instructs the JUnit Gradle plugin to set the
java.util.logging.manager system property to the supplied fully qualified class name
of the java.util.logging.LogManager implementation to use. The above example
demonstrates how to configure log4j as the LogManager.
By default, the JUnit Gradle plugin disables the standard Gradle test task, but this
can be overridden via the enableStandardTestTask flag.
Configuring Selectors
By default, the plugin will scan your project’s output directories for tests. However,
you can specify which tests to execute explicitly using the selectors extension element.
junitPlatform {
// ...
selectors {
uris 'file:///foo.txt', 'http://example.com/'
uri 'foo:resource' (1)
files 'foo.txt', 'bar.csv'
file 'qux.json' (2)
directories 'foo/bar', 'bar/qux'
directory 'qux/bar' (3)
packages 'com.acme.foo', 'com.acme.bar'
aPackage 'com.example.app' (4)
classes 'com.acme.Foo', 'com.acme.Bar'
aClass 'com.example.app.Application' (5)
methods 'com.acme.Foo#a', 'com.acme.Foo#b'
method 'com.example.app.Application#run(java.lang.String[])' (6)
resources '/bar.csv', '/foo/input.json'
resource '/com/acme/my.properties' (7)
}
// ...
}| 1 | URIs |
| 2 | Local files |
| 3 | Local directories |
| 4 | Packages |
| 5 | Classes, fully qualified class names |
| 6 | Methods, fully qualified method names (see selectMethod(String) in DiscoverySelectors) |
| 7 | Classpath resources |
Configuring Filters
You can configure filters for the test plan by using the filters extension. By default,
all engines and tags are included in the test plan. Only the default
includeClassNamePattern (^.*Tests?$) is applied. You can
override the default pattern as in the following example. When you specify multiple
patterns, they are combined using OR semantics.
junitPlatform {
// ...
filters {
engines {
include 'junit-jupiter'
// exclude 'junit-vintage'
}
tags {
include 'fast', 'smoke'
// exclude 'slow', 'ci'
}
packages {
include 'com.sample.included1', 'com.sample.included2'
// exclude 'com.sample.excluded1', 'com.sample.excluded2'
}
includeClassNamePattern '.*Spec'
includeClassNamePatterns '.*Test', '.*Tests'
}
// ...
}If you supply a Test Engine ID via engines {include …} or engines {exclude …},
the JUnit Gradle plugin will only run tests for the desired test engines. Similarly, if
you supply a tag via tags {include …} or tags {exclude …}, the JUnit Gradle
plugin will only run tests that are tagged accordingly (e.g., via the @Tag annotation
for JUnit Jupiter based tests). The same applies to package names that can be included or
excluded using packages {include …} or packages {exclude …}.
Configuration Parameters
You can set configuration parameters to influence test discovery and execution by using
the configurationParameter or configurationParameters DSL. The former can be used to
set a single configuration parameter, while the latter takes a map of configuration
parameters to set multiple key-value pairs at once. All keys and values must be Strings.
junitPlatform {
// ...
configurationParameter 'junit.jupiter.conditions.deactivate', '*'
configurationParameters([
'junit.jupiter.extensions.autodetection.enabled': 'true',
'junit.jupiter.testinstance.lifecycle.default': 'per_class'
])
// ...
}Configuring Test Engines
In order to have the JUnit Gradle plugin run any tests at all, a TestEngine
implementation must be on the classpath.
To configure support for JUnit Jupiter based tests, configure a testCompile dependency
on the JUnit Jupiter API and a testRuntime dependency on the JUnit Jupiter TestEngine
implementation similar to the following.
dependencies {
testCompile("org.junit.jupiter:junit-jupiter-api:5.0.0")
testRuntime("org.junit.jupiter:junit-jupiter-engine:5.0.0")
}The JUnit Gradle plugin can run JUnit 4 based tests as long as you configure a
testCompile dependency on JUnit 4 and a testRuntime dependency on the JUnit Vintage
TestEngine implementation similar to the following.
dependencies {
testCompile("junit:junit:4.12")
testRuntime("org.junit.vintage:junit-vintage-engine:4.12.0")
}Using the JUnit Gradle Plugin
Once the JUnit Gradle plugin has been applied and configured, you have a new
junitPlatformTest task at your disposal.
Invoking gradlew junitPlatformTest (or gradlew test) from the command line will
execute all tests within the project whose class names match the regular expression
supplied via the includeClassNamePattern (which defaults to
^.*Tests?$).
Executing the junitPlatformTest task in the junit5-gradle-consumer project results
in output similar to the following:
:junitPlatformTest Test run finished after 93 ms [ 3 containers found ] [ 0 containers skipped ] [ 3 containers started ] [ 0 containers aborted ] [ 3 containers successful ] [ 0 containers failed ] [ 3 tests found ] [ 1 tests skipped ] [ 2 tests started ] [ 0 tests aborted ] [ 2 tests successful ] [ 0 tests failed ] BUILD SUCCESSFUL
If a test fails, the build will fail with output similar to the following:
:junitPlatformTest
Test failures (1):
JUnit Jupiter:SecondTest:mySecondTest()
MethodSource [className = 'com.example.project.SecondTest', methodName = 'mySecondTest', methodParameterTypes = '']
=> Exception: 2 is not equal to 1 ==> expected: <2> but was: <1>
Test run finished after 99 ms
[ 3 containers found ]
[ 0 containers skipped ]
[ 3 containers started ]
[ 0 containers aborted ]
[ 3 containers successful ]
[ 0 containers failed ]
[ 3 tests found ]
[ 0 tests skipped ]
[ 3 tests started ]
[ 0 tests aborted ]
[ 2 tests successful ]
[ 1 tests failed ]
:junitPlatformTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':junitPlatformTest'.
> Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1
The exit value is 1 if any containers or tests failed; otherwise, it is 0.
|
|
Current Limitations of the JUnit Gradle Plugin
The results of any tests run via the JUnit Gradle plugin will not be included
in the standard test report generated by Gradle; however, the test results can
typically be aggregated on a CI server. See the reportsDir property of the plugin.
|
4.2.2. Maven
The JUnit team has developed a very basic provider for Maven Surefire that lets you run
JUnit 4 and JUnit Jupiter tests via mvn test. The pom.xml file in the
junit5-maven-consumer project demonstrates how to use it and can serve as a starting
point.
Due to a memory leak in Surefire 2.20, the junit-platform-surefire-provider
currently only works with Surefire 2.19.1.
|
...
<build>
<plugins>
...
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.0.0</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
...Configuring Test Engines
In order to have Maven Surefire run any tests at all, a TestEngine implementation must
be added to the runtime classpath.
To configure support for JUnit Jupiter based tests, configure a test dependency on the
JUnit Jupiter API, and add the JUnit Jupiter TestEngine implementation to the
dependencies of the maven-surefire-plugin similar to the following.
...
<build>
<plugins>
...
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.0.0</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
...
<dependencies>
...
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.0.0</version>
<scope>test</scope>
</dependency>
</dependencies>
...The JUnit Platform Surefire Provider can run JUnit 4 based tests as long as you configure
a test dependency on JUnit 4 and add the JUnit Vintage TestEngine implementation to
the dependencies of the maven-surefire-plugin similar to the following.
...
<build>
<plugins>
...
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.0.0</version>
</dependency>
...
<dependency>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
<version>4.12.0</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
...
<dependencies>
...
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
...Filtering by Tags
You can filter tests by tags using the following configuration properties.
-
to include a tag, use either
groupsorincludeTags -
to exclude a tag, use either
excludedGroupsorexcludeTags
...
<build>
<plugins>
...
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<properties>
<includeTags>acceptance</includeTags>
<excludeTags>integration, regression</excludeTags>
</properties>
</configuration>
<dependencies>
...
</dependencies>
</plugin>
</plugins>
</build>
...Configuration Parameters
You can set configuration parameters to influence test discovery and execution by using
the configurationParameters property and providing key-value pairs in the Java
Properties file syntax.
...
<build>
<plugins>
...
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<properties>
<configurationParameters>
junit.jupiter.conditions.deactivate = *
junit.jupiter.extensions.autodetection.enabled = true
junit.jupiter.testinstance.lifecycle.default = per_class
</configurationParameters>
</properties>
</configuration>
<dependencies>
...
</dependencies>
</plugin>
</plugins>
</build>
...4.3. Console Launcher
The ConsoleLauncher is a command-line Java application that lets you launch
the JUnit Platform from the console. For example, it can be used to run JUnit
Vintage and JUnit Jupiter tests and print test execution results to the console.
An executable junit-platform-console-standalone-1.0.0.jar with all
dependencies included is published in the central Maven repository under the
junit-platform-console-standalone
directory. You can run the
standalone ConsoleLauncher as shown below.
java -jar junit-platform-console-standalone-1.0.0.jar <Options>
Here’s an example of its output:
├─ JUnit Vintage
│ └─ example.JUnit4Tests
│ └─ standardJUnit4Test ✔
└─ JUnit Jupiter
├─ StandardTests
│ ├─ succeedingTest() ✔
│ └─ skippedTest() ↷ for demonstration purposes
└─ A special test case
├─ Custom test name containing spaces ✔
├─ ╯°□°）╯ ✔
└─ 😱 ✔
Test run finished after 64 ms
[ 5 containers found ]
[ 0 containers skipped ]
[ 5 containers started ]
[ 0 containers aborted ]
[ 5 containers successful ]
[ 0 containers failed ]
[ 6 tests found ]
[ 1 tests skipped ]
[ 5 tests started ]
[ 0 tests aborted ]
[ 5 tests successful ]
[ 0 tests failed ]
|
Exit Code
The ConsoleLauncher exits with a status code of 1 if any containers or tests
failed. Otherwise the exit code is 0.
|
4.3.1. Options
Option Description
------ -----------
-h, --help Display help information.
--disable-ansi-colors Disable ANSI colors in output (not
supported by all terminals).
--details <[none,flat,tree,verbose]> Select an output details mode for when
tests are executed. Use one of: [none,
flat, tree, verbose]. If 'none' is
selected, then only the summary and test
failures are shown. (default: tree)
--details-theme <[ascii,unicode]> Select an output details tree theme for
when tests are executed. Use one of:
[ascii, unicode] (default: unicode)
--class-path, --classpath, --cp <Path: Provide additional classpath entries --
path1:path2:...> for example, for adding engines and
their dependencies. This option can be
repeated.
--reports-dir <Path> Enable report output into a specified
local directory (will be created if it
does not exist).
--scan-class-path, --scan-classpath [Path: Scan all directories on the classpath or
path1:path2:...] explicit classpath roots. Without
arguments, only directories on the
system classpath as well as additional
classpath entries supplied via -cp
(directories and JAR files) are scanned.
Explicit classpath roots that are not on
the classpath will be silently ignored.
This option can be repeated.
-u, --select-uri <URI> Select a URI for test discovery. This
option can be repeated.
-f, --select-file <String> Select a file for test discovery. This
option can be repeated.
-d, --select-directory <String> Select a directory for test discovery.
This option can be repeated.
-p, --select-package <String> Select a package for test discovery. This
option can be repeated.
-c, --select-class <String> Select a class for test discovery. This
option can be repeated.
-m, --select-method <String> Select a method for test discovery. This
option can be repeated.
-r, --select-resource <String> Select a classpath resource for test
discovery. This option can be repeated.
-n, --include-classname <String> Provide a regular expression to include
only classes whose fully qualified names
match. To avoid loading classes
unnecessarily, the default pattern only
includes class names that end with
"Test" or "Tests". When this option is
repeated, all patterns will be combined
using OR semantics. (default: ^.*Tests?$)
-N, --exclude-classname <String> Provide a regular expression to exclude
those classes whose fully qualified
names match. When this option is
repeated, all patterns will be combined
using OR semantics.
--include-package <String> Provide a package to be included in the
test run. This option can be repeated.
--exclude-package <String> Provide a package to be excluded from the
test run. This option can be repeated.
-t, --include-tag <String> Provide a tag to be included in the test
run. This option can be repeated.
-T, --exclude-tag <String> Provide a tag to be excluded from the test
run. This option can be repeated.
-e, --include-engine <String> Provide the ID of an engine to be included
in the test run. This option can be
repeated.
-E, --exclude-engine <String> Provide the ID of an engine to be excluded
from the test run. This option can be
repeated.
--config <key=value> Set a configuration parameter for test
discovery and execution. This option can
be repeated.
4.4. Using JUnit 4 to Run the JUnit Platform
The JUnitPlatform runner is a JUnit 4 based Runner which enables you to run any test
whose programming model is supported on the JUnit Platform in a JUnit 4 environment — for example, a JUnit Jupiter test class.
Annotating a class with @RunWith(JUnitPlatform.class) allows it to be run with IDEs and
build systems that support JUnit 4 but do not yet support the JUnit Platform directly.
Since the JUnit Platform has features that JUnit 4 does not have, the runner is
only able to support a subset of the JUnit Platform functionality, especially with regard
to reporting (see Display Names vs. Technical Names). But for the
time being the JUnitPlatform runner is an easy way to get started.
|
4.4.1. Setup
You need the following artifacts and their dependencies on the classpath. See Dependency Metadata for details regarding group IDs, artifact IDs, and versions.
Explicit Dependencies
-
junit-4.12.jarin test scope: to run tests using JUnit 4. -
junit-platform-runnerin test scope: location of theJUnitPlatformrunner -
junit-jupiter-apiin test scope: API for writing tests, including@Test, etc. -
junit-jupiter-enginein test runtime scope: implementation of the Engine API for JUnit Jupiter
4.4.2. Display Names vs. Technical Names
By default, display names will be used for test artifacts; however, when the
JUnitPlatform runner is used to execute tests with a build tool such as Gradle or
Maven, the generated test report often needs to include the technical names of test
artifacts — for example, fully qualified class names — instead of shorter display names
like the simple name of a test class or a custom display name containing special
characters. To enable technical names for reporting purposes, simply declare the
@UseTechnicalNames annotation alongside @RunWith(JUnitPlatform.class).
4.4.3. Single Test Class
One way to use the JUnitPlatform runner is to annotate a test class with
@RunWith(JUnitPlatform.class) directly. Please note that the test methods in the
following example are annotated with org.junit.jupiter.api.Test (JUnit Jupiter), not
org.junit.Test (JUnit Vintage). Moreover, in this case the test class must be public;
otherwise, IDEs won’t recognize it as a JUnit 4 test class.
import static org.junit.jupiter.api.Assertions.fail;
import org.junit.jupiter.api.Test;
import org.junit.platform.runner.JUnitPlatform;
import org.junit.runner.RunWith;
@RunWith(JUnitPlatform.class)
public class JUnit4ClassDemo {
@Test
void succeedingTest() {
/* no-op */
}
@Test
void failingTest() {
fail("Failing for failing's sake.");
}
}4.4.4. Test Suite
If you have multiple test classes you can create a test suite as can be seen in the following example.
import org.junit.platform.runner.JUnitPlatform;
import org.junit.platform.suite.api.SelectPackages;
import org.junit.runner.RunWith;
@RunWith(JUnitPlatform.class)
@SelectPackages("example")
public class JUnit4SuiteDemo {
}The JUnit4SuiteDemo will discover and run all tests in the example package
and its subpackages. By default, it will only include test classes whose names match the
pattern ^.*Tests?$.
|
Additional Configuration Options
There are more configuration options for discovering and filtering tests than just
@SelectPackages. Please consult the
Javadoc for further
details.
|
4.5. Configuration Parameters
In addition to instructing the platform which test classes and test engines to include,
which packages to scan, etc., it is sometimes necessary to provide additional custom
configuration parameters that are specific to a particular test engine. For example, the
JUnit Jupiter TestEngine supports configuration parameters for the following use
cases.
Configuration Parameters are text-based key-value pairs that can be supplied to test engines running on the JUnit Platform via one of the following mechanisms.
-
The
configurationParameter()andconfigurationParameters()methods in theLauncherDiscoveryRequestBuilderwhich is used to build a request supplied to theLauncherAPI. When running tests via one of the tools provided by the JUnit Platform you can specify configuration parameters as follows:-
Console Launcher: use the
--configcommand-line option. -
Gradle plugin: use the
configurationParameterorconfigurationParametersDSL. -
Maven Surefire provider: use the
configurationParametersproperty.
-
-
JVM system properties.
-
The JUnit Platform configuration file: a file named
junit-platform.propertiesin the root of the class path that follows the syntax rules for a JavaPropertiesfile.
Configuration parameters are looked up in the exact order defined above.
Consequently, configuration parameters supplied directly to the Launcher take
precedence over those supplied via system properties and the configuration file.
Similarly, configuration parameters supplied via system properties take precedence over
those supplied via the configuration file.
|
5. Extension Model
5.1. Overview
In contrast to the competing Runner, @Rule, and @ClassRule extension points in
JUnit 4, the JUnit Jupiter extension model consists of a single, coherent concept: the
Extension API. Note, however, that Extension itself is just a marker interface.
5.2. Registering Extensions
Extensions can be registered explicitly via
@ExtendWith or automatically via Java’s
ServiceLoader mechanism.
5.2.1. Declarative Extension Registration
Developers can register one or more extensions declaratively by annotating a test
interface, test class, test method, or custom composed
annotation with @ExtendWith(…) and supplying class references for the extensions
to register.
For example, to register a custom MockitoExtension for a particular test method, you
would annotate the test method as follows.
@ExtendWith(MockitoExtension.class)
@Test
void mockTest() {
// ...
}To register a custom MockitoExtension for all tests in a particular class and its
subclasses, you would annotate the test class as follows.
@ExtendWith(MockitoExtension.class)
class MockTests {
// ...
}Multiple extensions can be registered together like this:
@ExtendWith({ FooExtension.class, BarExtension.class })
class MyTestsV1 {
// ...
}As an alternative, multiple extensions can be registered separately like this:
@ExtendWith(FooExtension.class)
@ExtendWith(BarExtension.class)
class MyTestsV2 {
// ...
}The execution of tests in both MyTestsV1 and MyTestsV2 will be extended by the
FooExtension and BarExtension, in exactly that order.
5.2.2. Automatic Extension Registration
In addition to declarative extension registration
support using annotations, JUnit Jupiter also supports global extension registration
via Java’s java.util.ServiceLoader mechanism, allowing third-party extensions to be
auto-detected and automatically registered based on what is available in the classpath.
Specifically, a custom extension can be registered by supplying its fully qualified class
name in a file named org.junit.jupiter.api.extension.Extension within the
/META-INF/services folder in its enclosing JAR file.
Enabling Automatic Extension Detection
Auto-detection is an advanced feature and is therefore not enabled by default. To enable
it, simply set the junit.jupiter.extensions.autodetection.enabled configuration
parameter to true. This can be supplied as a JVM system property, as a configuration
parameter in the LauncherDiscoveryRequest that is passed to the Launcher, or via the
JUnit Platform configuration file (see Configuration Parameters for details).
For example, to enable auto-detection of extensions, you can start your JVM with the following system property.
-Djunit.jupiter.extensions.autodetection.enabled=true
When auto-detection is enabled, extensions discovered via the ServiceLoader mechanism
will be added to the extension registry after JUnit Jupiter’s global extensions (e.g.,
support for TestInfo, TestReporter, etc.).
5.2.3. Extension Inheritance
Registered extensions are inherited within test class hierarchies with top-down semantics. Similarly, extensions registered at the class-level are inherited at the method-level. Furthermore, a specific extension implementation can only be registered once for a given extension context and its parent contexts. Consequently, any attempt to register a duplicate extension implementation will be ignored.
5.3. Conditional Test Execution
ExecutionCondition defines the Extension API for programmatic, conditional test
execution.
An ExecutionCondition is evaluated for each container (e.g., a test class) to
determine if all the tests it contains should be executed based on the supplied
ExtensionContext. Similarly, an ExecutionCondition is evaluated for each test to
determine if a given test method should be executed based on the supplied
ExtensionContext.
When multiple ExecutionCondition extensions are registered, a container or test is
disabled as soon as one of the conditions returns disabled. Thus, there is no guarantee
that a condition is evaluated because another extension might have already caused a
container or test to be disabled. In other words, the evaluation works like the
short-circuiting boolean OR operator.
See the source code of DisabledCondition and @Disabled for concrete examples.
5.3.1. Deactivating Conditions
Sometimes it can be useful to run a test suite without certain conditions being active.
For example, you may wish to run tests even if they are annotated with @Disabled in
order to see if they are still broken. To do this, simply provide a pattern for the
junit.jupiter.conditions.deactivate configuration parameter to specify which
conditions should be deactivated (i.e., not evaluated) for the current test run. The
pattern can be supplied as a JVM system property, as a configuration parameter in the
LauncherDiscoveryRequest that is passed to the Launcher, or via the JUnit Platform
configuration file (see Configuration Parameters for details).
For example, to deactivate JUnit’s @Disabled condition, you can start your JVM with the
following system property.
-Djunit.jupiter.conditions.deactivate=org.junit.*DisabledCondition
Pattern Matching Syntax
If the junit.jupiter.conditions.deactivate pattern consists solely of an asterisk
(*), all conditions will be deactivated. Otherwise, the pattern will be used to match
against the fully qualified class name (FQCN) of each registered condition. Any dot
(.) in the pattern will match against a dot (.) or a dollar sign ($) in the FQCN.
Any asterisk (*) will match against one or more characters in the FQCN. All other
characters in the pattern will be matched one-to-one against the FQCN.
Examples:
-
*: deactivates all conditions. -
org.junit.*: deactivates every condition under theorg.junitbase package and any of its subpackages. -
*.MyCondition: deactivates every condition whose simple class name is exactlyMyCondition. -
*System*: deactivates every condition whose simple class name containsSystem. -
org.example.MyCondition: deactivates the condition whose FQCN is exactlyorg.example.MyCondition.
5.4. Test Instance Post-processing
TestInstancePostProcessor defines the API for Extensions that wish to post
process test instances.
Common use cases include injecting dependencies into the test instance, invoking custom initialization methods on the test instance, etc.
For concrete examples, consult the source code for the MockitoExtension and the
SpringExtension.
5.5. Parameter Resolution
ParameterResolver defines the Extension API for dynamically resolving parameters at
runtime.
If a test constructor or a @Test, @TestFactory, @BeforeEach, @AfterEach,
@BeforeAll, or @AfterAll method accepts a parameter, the parameter must be resolved
at runtime by a ParameterResolver. A ParameterResolver can either be built-in (see
TestInfoParameterResolver) or registered by the user.
Generally speaking, parameters may be resolved by name, type, annotation, or any
combination thereof. For concrete examples, consult the source code for
CustomTypeParameterResolver and CustomAnnotationParameterResolver.
5.6. Test Lifecycle Callbacks
The following interfaces define the APIs for extending tests at various points in the
test execution lifecycle. Consult the following sections for examples and the Javadoc for
each of these interfaces in the org.junit.jupiter.api.extension package for further details.
|
Implementing Multiple Extension APIs
Extension developers may choose to implement any number of these interfaces
within a single extension. Consult the source code of the SpringExtension for a
concrete example.
|
5.6.1. Before and After Test Execution Callbacks
BeforeTestExecutionCallback and AfterTestExecutionCallback define the APIs for
Extensions that wish to add behavior that will be executed immediately before and
immediately after a test method is executed, respectively. As such, these callbacks are
well suited for timing, tracing, and similar use cases. If you need to implement
callbacks that are invoked around @BeforeEach and @AfterEach methods, implement
BeforeEachCallback and AfterEachCallback instead.
The following example shows how to use these callbacks to calculate and log the execution
time of a test method. TimingExtension implements both BeforeTestExecutionCallback
and AfterTestExecutionCallback in order to time and log the test execution.
import java.lang.reflect.Method;
import java.util.logging.Logger;
import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
import org.junit.jupiter.api.extension.BeforeTestExecutionCallback;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.ExtensionContext.Namespace;
import org.junit.jupiter.api.extension.ExtensionContext.Store;
public class TimingExtension implements BeforeTestExecutionCallback, AfterTestExecutionCallback {
private static final Logger LOG = Logger.getLogger(TimingExtension.class.getName());
@Override
public void beforeTestExecution(ExtensionContext context) throws Exception {
getStore(context).put(context.getRequiredTestMethod(), System.currentTimeMillis());
}
@Override
public void afterTestExecution(ExtensionContext context) throws Exception {
Method testMethod = context.getRequiredTestMethod();
long start = getStore(context).remove(testMethod, long.class);
long duration = System.currentTimeMillis() - start;
LOG.info(() -> String.format("Method [%s] took %s ms.", testMethod.getName(), duration));
}
private Store getStore(ExtensionContext context) {
return context.getStore(Namespace.create(getClass(), context));
}
}Since the TimingExtensionTests class registers the TimingExtension via @ExtendWith,
its tests will have this timing applied when they execute.
@ExtendWith(TimingExtension.class)
class TimingExtensionTests {
@Test
void sleep20ms() throws Exception {
Thread.sleep(20);
}
@Test
void sleep50ms() throws Exception {
Thread.sleep(50);
}
}The following is an example of the logging produced when TimingExtensionTests is run.
INFO: Method [sleep20ms] took 24 ms. INFO: Method [sleep50ms] took 53 ms.
5.7. Exception Handling
TestExecutionExceptionHandler defines the API for Extensions that wish to handle
exceptions thrown during test execution.
The following example shows an extension which will swallow all instances of IOException
but rethrow any other type of exception.
public class IgnoreIOExceptionExtension implements TestExecutionExceptionHandler {
@Override
public void handleTestExecutionException(ExtensionContext context, Throwable throwable)
throws Throwable {
if (throwable instanceof IOException) {
return;
}
throw throwable;
}
}5.8. Providing Invocation Contexts for Test Templates
A @TestTemplate method can only be executed when at least one
TestTemplateInvocationContextProvider is registered. Each such provider is responsible
for providing a Stream of TestTemplateInvocationContext instances. Each context may
specify a custom display name and a list of additional extensions that will only be used
for the next invocation of the @TestTemplate method.
The following example shows how to write a test template as well as how to register and
implement a TestTemplateInvocationContextProvider.
@TestTemplate
@ExtendWith(MyTestTemplateInvocationContextProvider.class)
void testTemplate(String parameter) {
assertEquals(3, parameter.length());
}
static class MyTestTemplateInvocationContextProvider implements TestTemplateInvocationContextProvider {
@Override
public boolean supportsTestTemplate(ExtensionContext context) {
return true;
}
@Override
public Stream<TestTemplateInvocationContext> provideTestTemplateInvocationContexts(ExtensionContext context) {
return Stream.of(invocationContext("foo"), invocationContext("bar"));
}
private TestTemplateInvocationContext invocationContext(String parameter) {
return new TestTemplateInvocationContext() {
@Override
public String getDisplayName(int invocationIndex) {
return parameter;
}
@Override
public List<Extension> getAdditionalExtensions() {
return Collections.singletonList(new ParameterResolver() {
@Override
public boolean supportsParameter(ParameterContext parameterContext,
ExtensionContext extensionContext) {
return parameterContext.getParameter().getType().equals(String.class);
}
@Override
public Object resolveParameter(ParameterContext parameterContext,
ExtensionContext extensionContext) {
return parameter;
}
});
}
};
}
}In this example, the test template will be invoked twice. The display names of the
invocations will be “foo” and “bar” as specified by the invocation context. Each
invocation registers a custom ParameterResolver which is used to resolve the method
parameter. The output when using the ConsoleLauncher is as follows.
└─ testTemplate(String) ✔ ├─ foo ✔ └─ bar ✔
The TestTemplateInvocationContextProvider extension API is primarily intended for
implementing different kinds of tests that rely on repetitive invocation of a test-like
method albeit in different contexts — for example, with different parameters, by preparing
the test class instance differently, or multiple times without modifying the context.
Please refer to the implementations of Repeated Tests or
Parameterized Tests which use this extension point to provide their
functionality.
5.9. Keeping State in Extensions
Usually, an extension is instantiated only once. So the question becomes relevant: How do
you keep the state from one invocation of an extension to the next? The
ExtensionContext API provides a Store exactly for this purpose. Extensions may put
values into a store for later retrieval. See the
TimingExtension for an example of
using the Store with a method-level scope. It is important to remember that values
stored in an ExtensionContext during test execution will not be available in the
surrounding ExtensionContext. Since ExtensionContexts may be nested, the scope of
inner contexts may also be limited. Consult the corresponding Javadoc for details on the
methods available for storing and retrieving values via the Store.
5.10. Supported Utilities in Extensions
The JUnit Platform Commons artifact exposes a package named
org.junit.platform.commons.support that contains maintained utility methods for working
with annotations, reflection, and classpath scanning tasks. TestEngine and Extension
authors are encouraged to use these supported methods in order to align with the behavior
of the JUnit Platform.
5.11. Relative Execution Order of User Code and Extensions
When executing a test class that contains one or more test methods, a number of extension callbacks are called in addition to the user-provided test and lifecycle methods. The following diagram illustrates the relative order of user-provided code and extension code.
User-provided test and lifecycle methods are shown in orange, with callback code provided by extensions shown in blue. The grey box denotes the execution of a single test method and will be repeated for every test method in the test class.
The following table further explains the twelve steps in the User code and extension code diagram.
| Step | Interface/Annotation | Description |
|---|---|---|
1 |
interface |
extension code executed before all tests of the container are executed |
2 |
annotation |
user code executed before all tests of the container are executed |
3 |
interface |
extension code executed before each test is executed |
4 |
annotation |
user code executed before each test is executed |
5 |
interface |
extension code executed immediately before a test is executed |
6 |
annotation |
user code of the actual test method |
7 |
interface |
extension code for handling exceptions thrown during a test |
8 |
interface |
extension code executed immediately after test execution and its corresponding exception handlers |
9 |
annotation |
user code executed after each test is executed |
10 |
interface |
extension code executed after each test is executed |
11 |
annotation |
user code executed after all tests of the container are executed |
12 |
interface |
extension code executed after all tests of the container are executed |
In the simplest case only the actual test method will be executed (step 6); all other steps are optional depending on the presence of user code or extension support for the corresponding lifecycle callback. For further details on the various lifecycle callbacks please consult the respective JavaDoc for each annotation and extension.
6. Migrating from JUnit 4
Although the JUnit Jupiter programming model and extension model will not support JUnit 4
features such as Rules and Runners natively, it is not expected that source code
maintainers will need to update all of their existing tests, test extensions, and custom
build test infrastructure to migrate to JUnit Jupiter.
Instead, JUnit provides a gentle migration path via a JUnit Vintage test engine which
allows existing tests based on JUnit 3 and JUnit 4 to be executed using the JUnit
Platform infrastructure. Since all classes and annotations specific to JUnit Jupiter
reside under a new org.junit.jupiter base package, having both JUnit 4 and JUnit
Jupiter in the classpath does not lead to any conflicts. It is therefore safe to maintain
existing JUnit 4 tests alongside JUnit Jupiter tests. Furthermore, since the JUnit team
will continue to provide maintenance and bug fix releases for the JUnit 4.x baseline,
developers have plenty of time to migrate to JUnit Jupiter on their own schedule.
6.1. Running JUnit 4 Tests on the JUnit Platform
Just make sure that the junit-vintage-engine artifact is in your test runtime path. In
that case JUnit 3 and JUnit 4 tests will automatically be picked up by the JUnit Platform
launcher.
See the example projects in the junit5-samples repository to
find out how this is done with Gradle and Maven.
6.2. Migration Tips
The following are things you have to watch out for when migrating existing JUnit 4 tests to JUnit Jupiter.
-
Annotations reside in the
org.junit.jupiter.apipackage. -
Assertions reside in
org.junit.jupiter.api.Assertions. -
Assumptions reside in
org.junit.jupiter.api.Assumptions. -
@Beforeand@Afterno longer exist; use@BeforeEachand@AfterEachinstead. -
@BeforeClassand@AfterClassno longer exist; use@BeforeAlland@AfterAllinstead. -
@Ignoreno longer exists: use@Disabledinstead. -
@Categoryno longer exists; use@Taginstead. -
@RunWithno longer exists; superseded by@ExtendWith. -
@Ruleand@ClassRuleno longer exist; superseded by@ExtendWith; see the following section for partial rule support.
6.3. Limited JUnit 4 Rule Support
As stated above, JUnit Jupiter does not and will not support JUnit 4 rules natively. The JUnit team realizes, however, that many organizations, especially large ones, are likely to have large JUnit 4 code bases that make use of custom rules. To serve these organizations and enable a gradual migration path the JUnit team has decided to support a selection of JUnit 4 rules verbatim within JUnit Jupiter. This support is based on adapters and is limited to those rules that are semantically compatible to the JUnit Jupiter extension model, i.e. those that do not completely change the overall execution flow of the test.
JUnit Jupiter currently supports the following three Rule types including subclasses of
those types:
-
org.junit.rules.ExternalResource(includingorg.junit.rules.TemporaryFolder) -
org.junit.rules.Verifier(includingorg.junit.rules.ErrorCollector) -
org.junit.rules.ExpectedException
As in JUnit 4, Rule-annotated fields as well as methods are supported. By using these
class-level extensions on a test class such Rule implementations in legacy code bases
can be left unchanged including the JUnit 4 rule import statements.
This limited form of Rule support can be switched on by the class-level annotation
org.junit.jupiter.migrationsupport.rules.EnableRuleMigrationSupport. This annotation is
a composed annotation which enables all migration support extensions:
VerifierSupport, ExternalResourceSupport, and ExpectedExceptionSupport.
However, if you intend to develop a new extension for JUnit 5 please use the new extension model of JUnit Jupiter instead of the rule-based model of JUnit 4.
JUnit 4 Rule support in JUnit Jupiter is currently an experimental feature.
Consult the table in Experimental APIs for detail.
|
7. Advanced Topics
7.1. JUnit Platform Launcher API
One of the prominent goals of JUnit 5 is to make the interface between JUnit and its programmatic clients – build tools and IDEs – more powerful and stable. The purpose is to decouple the internals of discovering and executing tests from all the filtering and configuration that’s necessary from the outside.
JUnit 5 introduces the concept of a Launcher that can be used to discover, filter, and
execute tests. Moreover, third party test libraries – like Spock, Cucumber, and FitNesse
– can plug into the JUnit Platform’s launching infrastructure by providing a custom
TestEngine.
The launching API is in the junit-platform-launcher module.
An example consumer of the launching API is the ConsoleLauncher in the
junit-platform-console project.
7.1.1. Discovering Tests
Introducing test discovery as a dedicated feature of the platform itself will (hopefully) free IDEs and build tools from most of the difficulties they had to go through to identify test classes and test methods in the past.
Usage Example:
import static org.junit.platform.engine.discovery.ClassNameFilter.includeClassNamePatterns;
import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;
import static org.junit.platform.engine.discovery.DiscoverySelectors.selectPackage;
import org.junit.platform.launcher.Launcher;
import org.junit.platform.launcher.LauncherDiscoveryRequest;
import org.junit.platform.launcher.TestExecutionListener;
import org.junit.platform.launcher.TestPlan;
import org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder;
import org.junit.platform.launcher.core.LauncherFactory;
import org.junit.platform.launcher.listeners.SummaryGeneratingListener;LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
.selectors(
selectPackage("com.example.mytests"),
selectClass(MyTestClass.class)
)
.filters(
includeClassNamePatterns(".*Tests")
)
.build();
Launcher launcher = LauncherFactory.create();
TestPlan testPlan = launcher.discover(request);There’s currently the possibility to select classes, methods, and all classes in a package or even search for all tests in the classpath. Discovery takes place across all participating test engines.
The resulting TestPlan is a hierarchical (and read-only) description of all engines,
classes, and test methods that fit the LauncherDiscoveryRequest. The client can
traverse the tree, retrieve details about a node, and get a link to the original source
(like class, method, or file position). Every node in the test plan has a unique ID
that can be used to invoke a particular test or group of tests.
7.1.2. Executing Tests
To execute tests, clients can use the same LauncherDiscoveryRequest as in the discovery
phase or create a new request. Test progress and reporting can be achieved by registering
one or more TestExecutionListener implementations with the Launcher as in the
following example.
LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
.selectors(
selectPackage("com.example.mytests"),
selectClass(MyTestClass.class)
)
.filters(
includeClassNamePatterns(".*Tests")
)
.build();
Launcher launcher = LauncherFactory.create();
// Register a listener of your choice
TestExecutionListener listener = new SummaryGeneratingListener();
launcher.registerTestExecutionListeners(listener);
launcher.execute(request);There is no return value for the execute() method, but you can easily use a listener to
aggregate the final results in an object of your own. For an example see the
SummaryGeneratingListener.
7.1.3. Plugging in Your Own Test Engine
JUnit currently provides two TestEngine implementations out of the box:
-
junit-jupiter-engine: The core of JUnit Jupiter. -
junit-vintage-engine: A thin layer on top of JUnit 4 to allow running vintage tests with the launcher infrastructure.
Third parties may also contribute their own TestEngine by implementing the interfaces
in the junit-platform-engine module and registering their engine. Engine registration
is currently supported via Java’s java.util.ServiceLoader mechanism. For example, the
junit-jupiter-engine module registers its org.junit.jupiter.engine.JupiterTestEngine
in a file named org.junit.platform.engine.TestEngine within the /META-INF/services in
the junit-jupiter-engine JAR.
7.1.4. Plugging in Your Own Test Execution Listeners
In addition to the public Launcher API method for registering test execution
listeners programmatically, custom TestExecutionListener implementations discovered
at runtime via Java’s java.util.ServiceLoader facility are automatically registered
with the DefaultLauncher. For example, an example.TestInfoPrinter class
implementing TestExecutionListener and declared within the
/META-INF/services/org.junit.platform.launcher.TestExecutionListener file is loaded
and registered automatically.
8. API Evolution
One of the major goals of JUnit 5 is to improve maintainers' capabilities to evolve JUnit despite its being used in many projects. With JUnit 4 a lot of stuff that was originally added as an internal construct only got used by external extension writers and tool builders. That made changing JUnit 4 especially difficult and sometimes impossible.
That’s why JUnit 5 introduces a defined lifecycle for all publicly available interfaces, classes, and methods.
8.1. API Version and Status
Every published artifact has a version number <major>.<minor>.<patch>, and all publicly
available interfaces, classes, and methods are annotated with @API from the
@API Guardian project. The annotation’s status attribute can be assigned one of the
following values.
| Status | Description |
|---|---|
|
Must not be used by any code other than JUnit itself. Might be removed without prior notice. |
|
Should no longer be used; might disappear in the next minor release. |
|
Intended for new, experimental features where we are looking for feedback. |
|
Intended for features that will not be changed in a backwards-
incompatible way for at least the next minor release of the current
major version. If scheduled for removal, it will be demoted to |
|
Intended for features that will not be changed in a backwards-
incompatible way in the current major version ( |
If the @API annotation is present on a type, it is considered to be applicable for all
public members of that type as well. A member is allowed to declare a different status
value of lower stability.
8.2. Experimental APIs
The following table lists which APIs are currently designated as experimental (via
@API(status = EXPERIMENTAL)). Caution should be taken when relying on such APIs.
| Package Name | Class Name | Type |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8.3. @API Tooling Support
The @API Guardian project plans to provide tooling support for publishers and consumers
of APIs annotated with @API. For example, the tooling support will likely provide a
means to check if JUnit APIs are being used in accordance with @API annotation
declarations.
9. Contributors
Browse the current list of contributors directly on GitHub.
10. Release Notes
5.0.0-M1
Date of Release: July 7, 2016
Scope: First milestone release of JUnit 5
Summary of Changes
The following is a list of global changes. For details regarding changes specific to the Platform, Jupiter, and Vintage, consult the dedicated subsections below. For a complete list of all closed issues and pull requests for this release, consult the 5.0 M1 milestone page in the JUnit repository on GitHub.
-
JAR manifests in published artifacts now contain additional metadata such as
Created-By,Built-By,Build-Date,Build-Time,Build-Revision,Implementation-Title,Implementation-Version,Implementation-Vendor, etc. -
Published artifacts now contain
LICENSE.mdinMETA-INF. -
JUnit now participates in the Up For Grabs movement for open source contributions.
-
See the up-for-grabs label on GitHub.
-
-
Group IDs, artifact IDs, and versions have changed for all published artifacts.
-
See Artifact Migration and Dependency Metadata.
-
-
All base packages have been renamed.
-
See Package Migration.
-
| Old Group ID | Old Artifact ID | New Group ID | New Artifact ID | New Base Version |
|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Old Base Package | New Base Package |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
JUnit Platform
-
The
ConsoleRunnerhas been renamed toConsoleLauncher. -
ConsoleLaunchernow always returns the status code on exit, and the enable exit code flags have been removed. -
The
junit-platform-consoleartifact no longer defines transitive dependencies onjunit-platform-runner,junit-jupiter-engine, orjunit-vintage-engine. -
The
JUnit5Runner has been renamed toJUnitPlatform.-
@Packageshas been renamed to@SelectPackages. -
@Classeshas been renamed to@SelectClasses. -
@UniqueIdshas been removed. -
@UseTechnicalNameshas been introduced.
-
-
The Gradle plugin for the JUnit Platform has been completely overhauled.
-
The JUnit Platform Gradle plugin now requires Gradle 2.5 or higher.
-
The
junit5TestGradle task has been renamed tojunitPlatformTest. -
The
junit5Gradle plugin configuration has been renamed tojunitPlatform.-
runJunit4has been replaced byenableStandardTestTask. -
versionhas been replaced byplatformVersion.
-
-
See Gradle for further details.
-
-
XML test report generation has been overhauled.
-
XML reports now contain newlines.
-
Attributes specific to the JUnit Platform that do not align with standard attributes in the de facto standard XML schema are now contained in
CDATAblocks within the<system-out>element. -
XML reports now use real method names and fully qualified class names instead of display names.
-
-
Unique ID in
TestIdentifieris now aString. -
TestSourceis now an interface with a dedicated hierarchy consisting ofCompositeTestSource,JavaSource,JavaPackageSource,JavaClassSource,JavaMethodSource,UriSource,FileSystemSource,DirectorySource, andFileSource. -
All
DiscoverySelectorfactory methods have been moved to a newDiscoverySelectorsclass that serves as a centralized collection of all select methods. -
Filter.filter()has been renamed toFilter.apply(). -
TestTag.of()has been renamed toTestTag.create(). -
TestDiscoveryRequesthas been renamed toLauncherDiscoveryRequest. -
TestDiscoveryRequestBuilderhas been renamed toLauncherDiscoveryRequestBuilder. -
LauncherDiscoveryRequestis now immutable. -
TestDescriptor.allDescendants()has been renamed toTestDescriptor.getAllDescendants(). -
TestEngine#discover(EngineDiscoveryRequest)has been replaced byTestEngine#discover(EngineDiscoveryRequest, UniqueId). -
Introduced
ConfigurationParameterswhich theLaunchersupplies to engines via theEngineDiscoveryRequestandExecutionRequest -
The
ContainerandLeafabstractions have been removed from theHierarchicalTestEngine. -
The
getName()method has been removed fromTestIdentifierandTestDescriptorin favor of retrieving an implementation specific name via theTestSource. -
Test engines are now permitted to be completely dynamic in nature. In other words, a
TestEngineis no longer required to createTestDescriptorentries during the discovery phase; aTestEnginemay now optionally register containers and tests dynamically during the execution phase. -
Include and exclude support for engines and tags has been completely revised.
-
Engines and tags can no longer be required but rather included.
-
ConsoleLaunchernow supports the following options:t/include-tag,T/exclude-tag,e/include-engine,E/exclude-engine. -
The Gradle plugin now supports
enginesandtagsconfiguration blocks with nestedincludeandexcludeentries. -
EngineFilternow supportsincludeEngines()andexcludeEngines()factory methods. -
The
JUnitPlatformrunner now supports@IncludeTags,@ExcludeTags,@IncludeEngines, and@ExcludeEngines.
-
JUnit Jupiter
-
The
junit5engine ID has been renamed tojunit-jupiter. -
JUnit5TestEnginehas been renamed toJupiterTestEngine. -
Assertionsnow provides the following support:-
assertEquals()for primitive types -
assertEquals()for doubles and floats with deltas -
assertArrayEquals() -
Expected and actual values are now supplied to the
AssertionFailedError.
-
-
Dynamic Tests: tests can now be registered dynamically at runtime via lambda expressions.
-
TestInfonow provides access to tags viagetTags(). -
@AfterEachmethods and after callbacks are now invoked if an exception is thrown by a@Testmethod, a@BeforeEachmethod, or a before callback. -
@AfterAllmethods and after all callbacks are now guaranteed to be invoked. -
Repeatable annotations such as
@ExtendWithand@Tagare now discovered in superclasses within a test class hierarchy as well as on interfaces. -
Extensions are now registered top-down within a test class or interface hierarchy.
-
Test and container execution conditions can now be deactivated.
-
InstancePostProcessorhas been renamed toTestInstancePostProcessor.-
TestInstancePostProcessorimplementations are now properly applied within@Nestedtest class hierarchies.
-
-
MethodParameterResolverhas been renamed toParameterResolver.-
The
ParameterResolverAPI is now based onjava.lang.reflect.Executableand can therefore be used to resolve parameters for methods and constructors. -
New
ParameterContextwhich is passed to thesupports()andresolve()methods ofParameterResolverextensions. -
Resolution of primitive types is now supported for
ParameterResolverextensions.
-
-
The
ExtensionPointRegistryandExtensionRegistrarhave been removed in favor of declarative registration via@ExtendWith. -
BeforeAllExtensionPointhas been renamed toBeforeAllCallback. -
AfterAllExtensionPointhas been renamed toAfterAllCallback. -
BeforeEachExtensionPointhas been renamed toBeforeEachCallback. -
BeforeAllExtensionPointhas been renamed toBeforeAllCallback. -
New
BeforeTestExecutionCallbackandAfterTestExecutionCallbackextension APIs. -
ExceptionHandlerExtensionPointhas been renamed toTestExecutionExceptionHandler. -
Test exceptions are now supplied to extensions via the
TestExtensionContext. -
ExtensionContext.Storenow supports type-safe variants of many of its methods. -
ExtensionContext.getElement()now returns anOptional. -
Namespace.of()has been renamed toNamespace.create(). -
TestInfoandExtensionContexthave newgetTestClass()andgetTestMethod()methods. -
The
getName()method has been removed fromTestInfoandExtensionContextin favor of retrieving a context specific name via the current test class or test method.
5.0.0-M2
Date of Release: July 23, 2016
Scope: Second milestone release of JUnit 5
Summary of Changes
This release is primarily a bugfix release for bugs discovered since 5.0.0-M1.
The following is a list of global changes. For details regarding changes specific to the Platform, Jupiter, and Vintage, consult the dedicated subsections below. For a complete list of all closed issues and pull requests for this release, consult the 5.0 M2 milestone page in the JUnit repository on GitHub.
-
The JUnit 5 Gradle build now runs properly on Microsoft Windows.
-
A continuous integration build against Microsoft Windows has been set up on AppVeyor.
JUnit Platform
Bug Fixes
-
Failures in containers — for example,
@BeforeAllmethods that throw exceptions — now fail the build when using theConsoleLauncheror the JUnit Platform Gradle plugin. -
The JUnit Platform Surefire Provider no longer silently ignores purely dynamic test classes — for example, test classes that only declare
@TestFactorymethods. -
The
junit-platform-consoleandjunit-platform-console.batshell scripts included in thejunit-platform-console-<release version>TAR and ZIP distributions now properly refer to theConsoleLauncherinstead of theConsoleRunner. -
The
TestExecutionSummaryused by theConsoleLauncherand the JUnit Platform Gradle plugin now includes the actual exception type for failures. -
Classpath scanning is now safeguarded against exceptions encountered during class loading and processing — for example, when processing classes with malformed names. The underlying exception is swallowed and logged along with the offending file path. If the exception is a blacklisted exception such as an
OutOfMemoryError, however, it will be rethrown.
5.0.0-M3
Date of Release: November 30, 2016
Scope: Third milestone release of JUnit 5 with a focus on JUnit 4 interoperability, additional discovery selectors, and documentation.
For a complete list of all closed issues and pull requests for this release, consult the 5.0 M3 milestone page in the JUnit repository on GitHub.
JUnit Platform
Bug Fixes
-
ColoredPrintingTestListener, which is used by theConsoleLauncher, now outputs the actual exception type and its stack trace when printing an exception message. -
Test classes in the default package are now picked up via classpath scanning when scanning classpath roots — for example, in conjunction with the JUnit Platform Gradle plugin when no explicit packages have been selected.
-
Classpath scanning no longer loads classes that are excluded by a class name filter.
-
Classpath scanning no longer attempts to load Java 9
module-info.classfiles.
Deprecations and Breaking Changes
-
ClasspathSelectorhas been renamed toClasspathRootSelectorto avoid confusion withClasspathResourceSelector. -
JavaPackageSourcehas been renamed toPackageSourceto align withPackageSelector. -
JavaClassSourcehas been renamed toClassSourceto align withClassSelector. -
JavaMethodSourcehas been renamed toMethodSourceto align withMethodSelector. -
PackageSource,ClassSource, andMethodSourcenow directly implement theTestSourceinterface instead of theJavaSourceinterface which has been removed. -
Generic name-based discovery selectors (i.e.,
selectName()andselectNames()) inDiscoverySelectorshave been deprecated in favor of the dedicatedselectPackage(String),selectClass(String), andselectMethod(String)methods. -
ClassFilterhas been renamed toClassNameFilterand now implementsDiscoveryFilter<String>instead ofDiscoveryFilter<Class<?>>so it can be applied before loading classes during classpath scanning. -
ClassNameFilter.includeClassNamePatternis now deprecated in favor ofClassNameFilter.includeClassNamePatterns. -
@IncludeClassNamePatternis now deprecated in favor of@IncludeClassNamePatterns. -
The
-pcommand-line option for configuring additional classpath entries for theConsoleLauncherhas been renamed to-cpin order to align with the option names for the standardjavaexecutable. In addition, a new--class-pathalias has been introduced, while the existing--classpathcommand-line option remains unchanged. -
The
-aand--allcommand-line options for theConsoleLauncherhave been renamed to--scan-class-path. -
The
--xml-reports-dircommand-line option for theConsoleLauncherhas been renamed to--reports-dir. -
The
-C,-D, and-rshort command-line options for theConsoleLauncherhave been removed in favor of using their long equivalents--disable-ansi-colors,--hide-details, and--reports-dirrespectively. -
The
ConsoleLauncherno longer supports passing non-option arguments. Please use the new explicit selector options to specify package, class, or method names instead. Alternatively,--scan-class-pathnow accepts an optional argument that can be used to select classpath entries to be scanned. Multiple classpath entries may be used by separating them using the system’s path separator (;on Windows,:on Unix). -
LauncherDiscoveryRequestBuilderno longer acceptsnullvalues forselectors,filters, or maps of configuration parameters. -
Filters for class name, engine IDs, and tags in the Gradle plugin configuration now need to be wrapped in a
filtersextension element (see Configuring Filters).
New Features
-
New
selectUri(…)methods inDiscoverySelectorsfor selecting URIs. ATestEnginecan retrieve such values by querying registered instances ofUriSelector. -
New
selectFile(…)andselectDirectory(…)methods inDiscoverySelectorsfor selecting files and directories in the file system. ATestEnginecan retrieve such values by querying registered instances ofFileSelectorandDirectorySelector. -
New
selectClasspathResource(String)method inDiscoverySelectorsfor selecting classpath resources such as XML or JSON files by name, where the name is a/-separated path name for the resource within the current classpath. ATestEnginecan retrieve such values by querying registered instances ofClasspathResourceSelector. Furthermore, a classpath resource can be made available as theTestSourceof aTestIdentifiervia the newClasspathResourceSource. -
The
selectMethod(String)method inDiscoverySelectorsnow supports selection of a fully qualified method name for a method that accepts parameters — for example,"org.example.TestClass#testMethod(org.junit.jupiter.api.TestInfo)". -
The
TestExecutionSummaryused by theConsoleLauncherand the JUnit Platform Gradle plugin now includes statistics for all container events in addition to test events. -
The
TestExecutionSummarycan now be used to get the list of all failures. -
The
ConsoleLauncher, the Gradle plugin, and theJUnitPlatformrunner now use^.*Tests?$as the default pattern for class names to be included in the test run. -
The Gradle plugin now allows to explicitly select which tests should be executed (see Configuring Selectors).
-
New
@Testableannotation thatTestEngineimplementations can use to signal to IDEs and tooling vendors that the annotated or meta-annotated element is testable (i.e. that it can be executed as a test on the JUnit Platform). -
Multiple regular expressions that are combined using OR semantics can now be passed to
ClassNameFilter.includeClassNamePatterns. -
Multiple regular expressions that are combined using OR semantics can now be passed via
@IncludeClassNamePatternsto theJUnitPlatformrunner. -
Multiple regular expressions that are combined using OR semantics can now be passed to the JUnit Platform Gradle plugin (see Configuring Filters) and the
ConsoleLauncher(see Options). -
Package names can now be included or excluded using
PackageNameFilter.includePackageNamesorPackageNameFilter.excludePackageNames. -
Package names can now be included or excluded using
@IncludePackagesand@ExcludePackagesin combination with theJUnitPlatformrunner. -
Package names can now be included or excluded via a filter configuration to the JUnit Platform Gradle plugin (see Configuring Filters) and the
ConsoleLauncher(see Options). -
The
junit-platform-consoleno longer has a mandatory dependency on JOpt Simple. Thus, it’s now possible to test custom code that uses a different version of that library. -
Resolution of package selectors now also scans JAR files.
-
The Surefire provider now supports forking.
-
The Surefire provider now supports filtering by tags using one of the following parameters:
-
include:
groups/includeTags -
exclude:
excludedGroups/excludeTags
-
-
The Surefire provider is now licensed under the Apache License v2.0.
JUnit Jupiter
Bug Fixes
-
@AfterEachmethods are now executed with bottom-up semantics within a test class hierarchy. -
DynamicTest.stream()now accepts aThrowingConsumerinstead of a conventionalConsumerfor its test executor, thereby allowing for custom streams of dynamic tests that may potentially throw checked exceptions. -
Extensions registered at the test method level are now used when invoking
@BeforeEachand@AfterEachmethods for the corresponding test method. -
The
JupiterTestEnginenow supports selection of test methods via their unique ID for methods that accept arrays or primitive types as parameters. -
ExtensionContext.Storeis now thread-safe.
Deprecations and Breaking Changes
-
The
Executablefunctional interface has been relocated to a new dedicatedorg.junit.jupiter.api.functionpackage. -
Assertions.expectThrows()has been deprecated in favor ofAssertions.assertThrows().
New Features and Improvements
-
Support for lazy and preemptive timeouts with lambda expressions in
Assertions. See examples inAssertionsDemoand consult theorg.junit.jupiter.AssertionsJavadoc for further details. -
New
assertIterableEquals()assertion that checks that two Iterables are deeply equal (see Javadoc for details). -
New variants of
Assertions.assertAll()that accept streams of executables (i.e.,Stream<Executable>). -
Assertions.assertThrows()now returns the thrown exception. -
@BeforeAlland@AfterAllmay now be declared on static methods in interfaces. -
The JUnit 4 Rules
org.junit.rules.ExternalResource,org.junit.rules.Verifier,org.junit.rules.ExpectedExceptionincluding subclasses are now supported in JUnit Jupiter to facilitate migration of JUnit 4 codebases.
5.0.0-M4
Date of Release: April 1, 2017
Scope: Fourth milestone release of JUnit 5 with a focus on test templates, repeated tests, and parameterized tests.
For a complete list of all closed issues and pull requests for this release, consult the 5.0 M4 milestone page in the JUnit repository on GitHub.
JUnit Platform
Bug Fixes
-
The JUnit Platform Gradle plugin now adds its dependencies with a fixed version (same as the plugin version) instead of a dynamic versioning scheme (was
1.+) by default to ensure reproducible builds. -
The JUnit Platform Gradle plugin now explicitly applies the Gradle built-in
javaplugin as it has an implicit dependency on it being applied. -
All
findMethods()implementations inReflectionUtilsno longer return synthetic methods. Shadowed override-equal methods are also no longer included in the results. -
Introduced
getLegacyReportingName()inTestIdentifierandTestDescriptor. This allows the JUnit Platform Gradle Plugin and the Surefire Provider to resolve the class and method names throughgetLegacyReportingName()instead of using the source location. -
JAR files that contain spaces in their paths (
%20) are now properly decoded before being used in classpath scanning routines. -
Updated
findNestedClasses()inReflectionUtilsso that searches for nested classes also return inherited nested classes (regardless of whether they are static or not).
Deprecations and Breaking Changes
-
All test suite annotations from the
org.junit.platform.runnerpackage have been moved to theorg.junit.platform.suite.apipackage in the newjunit-platform-suite-apimodule. This includes annotations such as@SelectClasses,@SelectPackages, etc. -
Removed deprecated
selectNames()name-based discovery selector fromDiscoverySelectorsin favor of the dedicatedselectPackage(String),selectClass(String), andselectMethod(String)methods. -
ClassNameFilter.includeClassNamePatternwas removed; please useClassNameFilter.includeClassNamePatternsinstead. -
@IncludeClassNamePatternhas been removed; please use@IncludeClassNamePatternsinstead. -
The
--hide-detailsoption of theConsoleLauncheris deprecated; use--details noneinstead. -
The ZIP distribution containing the console launcher is no longer provided. It has been replaced by an executable standalone JAR distribution. For details, see the "New Features" section below.
-
The
MethodSortOrderenum inReflectionUtilshas been renamed toHierarchyTraversalMode. Those affected should now useReflectionSupportinstead ofReflectionUtils. -
The method
execute(LauncherDiscoveryRequest launcherDiscoveryRequest)inLauncherhas been deprecated and will be removed in milestone M5. Instead use the following new method that registers suppliedTestExecutionListeners in addition to already registered listeners but only for the suppliedLauncherDiscoveryRequest:execute(LauncherDiscoveryRequest launcherDiscoveryRequest, TestExecutionListener… listeners)
New Features and Improvements
-
Custom
TestExecutionListenerimplementations can now be registered automatically via Java’sServiceLoadermechanism. -
New
getGroupId(),getArtifactId(), andgetVersion()default methods in theTestEngineAPI for debugging and reporting purposes. By default, package attributes (i.e., typically from JAR manifest attributes) are used to determine the artifact ID and version; whereas, the group ID is empty by default. Consult the Javadoc forTestEnginefor details. -
Logging information for discovered test engines has been enhanced to include the group ID, artifact ID, and version of each test engine if available via the
getGroupId(),getArtifactId(), andgetVersion()methods. -
The
--scan-classpathoption of theConsoleLaunchernow allows to scan JAR files for tests when they are supplied as explicit arguments (see Options). -
The new
--details <Details>option of theConsoleLaunchernow allows to select an output details mode for when tests are executed. Use one of:none,flat,treeorverbose. Ifnoneis selected, then only the summary and test failures are shown (see Options). -
The new
--details-theme <Theme>option of theConsoleLaunchernow allows to select a theme for when tests are executed and printed as a tree. Use one of:asciiorunicode(see Options). -
An executable
junit-platform-console-standalone-1.0.0.jarartifact containing the Jupiter and Vintage test engines with all dependencies is generated by the default build process, stored injunit-platform-console-standalone/build/libs, and published to Maven Central. It provides hassle-free usage of JUnit 5 in projects that manually manage their dependencies similar to the plain-old JAR known from JUnit 4. -
New
--exclude-classname(--N) option added to theConsoleLauncheraccepting a regular expression to exclude those classes whose fully qualified names match. When this option is repeated, all patterns will be combined using OR semantics. -
New JUnit Platform support package
org.junit.platform.commons.supportthat contains maintained utility methods for annotation, reflection, and classpath scanning tasks.TestEngineandExtensionauthors are encouraged to use these supported methods in order to align with the behavior of the JUnit Platform. -
Squashed logic behind
TestDescriptor.isTest()andTestDescriptor.isContainer()from two independentbooleanproperties into a single enumeration, namelyTestDescriptor.Type. See next bullet point for details. -
Introduced
TestDescriptor.Typeenumeration withTestDescriptor.getType()accessor defining all possible descriptor types.TestDescriptor.isTest()andTestDescriptor.isContainer()now delegate toTestDescriptor.Typeconstants. -
Introduced
TestDescriptor.prune()andTestDescriptor.pruneTree()which allow engine authors to customize what happens when pruning is triggered by the JUnit Platform. -
TestIdentifiernow uses the newTestDescriptor.Typeenumeration to store the underlying type. It can be retrieved via the newTestIdentifier.getType()method. Furthermore,TestIdentifier.isTest()andTestIdentifier.isContainer()now delegate toTestDescriptor.Typeconstants.
JUnit Jupiter
Bug Fixes
-
Fixed bug that prevented discovery of two or more methods in the same class when selected via a method selector.
-
@Nestednon-static test classes are now detected when declared in super classes. -
The correct execution order of overridden
@BeforeEachand@AfterEachmethods is now enforced when declared at multiple levels within a class hierarchy. It’s now alwayssuper.before,this.before,this.test,this.after, andsuper.after, even if the compiler adds synthetic methods. -
TestExecutionExceptionHandlers are now invoked in the opposite order in which they were registered, analogous to all other "after" extensions.
Deprecations and Breaking Changes
-
Removed deprecated
Assertions.expectThrows()method in favor ofAssertions.assertThrows(). -
ExtensionContext.Namespaces composed of the same parts but in a different order are no longer considered equal to each other.
New Features and Improvements
-
First-class support for parameterized tests via the new
@ParameterizedTestannotation. See Parameterized Tests for details. -
First-class support for repeated tests via the new
@RepeatedTestannotation andRepetitionInfoAPI. See Repeated Tests for details. -
Introduce
@TestTemplateannotation and accompanying extension pointTestTemplateInvocationContextProvider. -
Assertions.assertThrows()now uses canonical names for exception types when generating assertion failure messages. -
TestInstancePostProcessors registered on test methods are now invoked. -
New variants of
Assertions.fail:Assertions.fail(Throwable cause)andAssertions.fail(String message, Throwable cause). -
New
Assertions.assertLinesMatch()comparing lists of strings, featuringObject::equalsand regular expression checks.assertLinesMatch()also provides a fast-forward mechanism to skip lines that are expected to change in each invocation — for example, duration, timestamps, stack traces, etc. Consult the JavaDoc fororg.junit.jupiter.Assertionsfor details. -
Extensions can now be registered automatically via Java’s
ServiceLoadermechanism. See Automatic Extension Registration.
5.0.0-M5
Date of Release: July 4, 2017
Scope: Fifth milestone release of JUnit 5 with a focus on dynamic containers, test instance lifecycle management, and minor API changes.
| This is a milestone release and contains breaking changes. Please refer to the instructions above to use this version in a version of IntelliJ IDEA that bundles an older milestone release. |
For a complete list of all closed issues and pull requests for this release, consult the 5.0 M5 milestone page in the JUnit repository on GitHub.
-
All published JAR artifacts now contain an
Automatic-Module-Namemanifest attribute whose value is used as the name of the automatic module defined by that JAR file when it is placed on the Java 9 module path.
| JAR file | Automatic-Module-Name |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
JUnit Platform
Bug Fixes
-
MethodSelector.getMethodParameterTypes()no longer returnsnullfor parameter types if the selector is created viaDiscoverySelectorswithout explicit parameter types. Specifically, if parameter types are not supplied and cannot be deduced, an empty string will be returned. Similarly, if parameter types are not explicitly supplied but can be deduced (e.g, via a suppliedMethodreference),getMethodParameterTypes()now returns a string containing the deduced parameter types. -
Instead of throwing a
NullPointerExceptioninDetails.TREEmode, theConsoleLaunchernow prints the corresponding throwable’stoString()representation as a fallback mechanism. -
DefaultLaunchernow catches and warns about exceptions generated by an engine in its discovery and execution phases. Other engines are processed normally. -
UniqueId.Segmenttype and value strings are now partially URL encoded when the string representation of a unique ID is generated. All characters reserved by the activeUniqueIdFormatsyntax (e.g.,[,:,], and/) are encoded. The default parser has also been updated to decode such encoded segments.
Deprecations and Breaking Changes
-
The deprecated
--hide-detailsoption of theConsoleLauncherhas been removed; use--details noneinstead. -
The following previously deprecated methods have been removed.
-
junit-platform-engine:Node.execute(EngineExecutionContext) -
junit-platform-commons:ReflectionUtils.findAllClassesInClasspathRoot(Path, Predicate, Predicate)
-
-
The
isLeaf()method of theorg.junit.platform.engine.support.hierarchical.Nodeinterface has been removed. -
The default methods
pruneTree()andhasTests()have been removed fromTestDescriptor.
New Features and Improvements
-
When using
DiscoverySelectorsto select a method by its fully qualified method name or by providingmethodParameterTypesas a comma-separated string, it is now possible to describe array parameter types using source code syntax such asint[]for a primitive array andjava.lang.String[]for an object array. Furthermore, it is now possible to describe multidimensional array types using either the JVM’s internal String representation (e.g.,[[[Iforint[][][],[[Ljava.lang.String;forjava.lang.String[][], etc.) or source code syntax (e.g.,boolean[][][],java.lang.Double[][], etc.). -
The JUnit Platform Gradle plugin task
junitPlatformTestcan now be accessed directly during the configuration phase of a build. -
The JUnit Platform Gradle plugin now plays nice with the Gradle Kotlin DSL.
-
When a
TestEngineis discovered via Java’sServiceLoadermechanism, an attempt will now be made to determine the location from which the engine class was loaded, and if the location URL is available, it will be logged at configuration-level. -
The
mayRegisterTests()method may be used to signal that aTestDescriptorwill register dynamic tests during execution. -
The
TestPlanmay now be queried for whether itcontainsTests(). This is used by the Surefire provider to decide if a class is a test class that should be executed. -
The
ENGINEenum constant was removed fromTestDescriptor.Type. The default type of theEngineDescriptoris nowTestDescriptor.Type.CONTAINER.
JUnit Jupiter
Bug Fixes
-
@ParameterizedTestarguments are no longer resolved for lifecycle methods and test class constructors to improve interoperability with regular and parameterized test methods with different parameter lists.
Deprecations and Breaking Changes
-
The migration support module is now named
junit-jupiter-migrationsupport, without the dash-betweenmigrationandsupport. -
In order to ensure the composability of all supported extension APIs in JUnit Jupiter, several methods in existing APIs have been renamed.
-
See Extension API Migration for details.
-
-
The
arguments()method in theArgumentsProviderAPI in thejunit-jupiter-paramsmodule has been renamed toprovideArguments(). -
The
ObjectArrayArgumentsclass in thejunit-jupiter-paramsmodule has been removed; the functionality for creating anArgumentsinstance is now available via theArguments.of(…)static factory method. -
The
namesproperty of@MethodSourcehas been renamed tovalue. -
The
getTestInstance()method in theTestExtensionContextAPI has been moved to theExtensionContextAPI. Furthermore, the signature has changed fromObject getTestInstance()toOptional<Object> getTestInstance(). -
The
getTestException()method in theTestExtensionContextAPI has been moved to theExtensionContextAPI and renamed togetExecutionException(). -
The
TestExtensionContextandContainerExtensionContextinterfaces have been removed, and allExtensioninterfaces have been changed to useExtensionContextinstead. -
TestExecutionConditionandContainerExecutionConditionhave been replaced by a single, general purpose extension API for conditional test execution:ExecutionCondition.
| Extension API | Old Name | New Name/Location |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
New Features and Improvements
-
The test instance lifecycle can now be switched from the default per-method mode to a new per-class mode via the new class-level
@TestInstanceannotation. This enables shared test instance state between test methods in a given test class as well as between non-static@BeforeAlland@AfterAllmethods in the test class.-
See Test Instance Lifecycle for details.
-
-
@BeforeAlland@AfterAllmethods are no longer required to bestaticif the test class is annotated with@TestInstance(Lifecycle.PER_CLASS). This enables the following new features.-
Declaration of
@BeforeAlland@AfterAllmethods in@Nestedtest classes. -
Declaration of
@BeforeAlland@AfterAllon interfacedefaultmethods. -
Simplified declaration of
@BeforeAlland@AfterAllmethods in test classes implemented with the Kotlin programming language.
-
-
Assertions.assertAll()now tracks exceptions of any type (as opposed to only tracking exceptions of typeAssertionError), unless the exception is a blacklisted exception in which case it will be immediately rethrown. -
If a
@ParameterizedTestaccepts an array as an argument, the string representation of the array will now be converted to a human readable format when generating the display name for invocations of the parameterized test. -
The
@EnumSourcenow provides an enum constant selection mode that controls how the supplied names are interpreted. Supported modes includeINCLUDEandEXCLUDEas well as regular expression pattern matching modesMATCH_ALLandMATCH_ANY. -
Extensions may now share state across top-level test classes by using the
Storeof the newly introduced engine-levelExtensionContext. -
Argument providing methods referenced using
@MethodSourcemay now return instances ofDoubleStream,IntStream, andLongStreamdirectly. -
@TestFactorynow supports arbitrarily nested dynamic containers. SeeDynamicContainerand abstract base classDynamicNodefor details. -
ExtensionContext.getExecutionException()now provides exceptions thrown in@BeforeAllmethods orBeforeAllCallbackstoAfterAllCallbacks.
JUnit Vintage
Bug Fixes
-
The
VintageTestEngineno longer filters out test classes declared as static member classes, since they are valid JUnit 4 test classes. -
The
VintageTestEngineno longer attempts to execute abstract classes as test classes. Instead, a warning is now logged stating that such classes are excluded.
5.0.0-M6
Date of Release: July 18, 2017
Scope: Sixth milestone release of JUnit 5 with a focus on Java 9 compatibility, validation (e.g. tag syntax rules), and bug fixes.
| This is a milestone release and contains breaking changes. Please refer to the instructions above to use this version in a version of IntelliJ IDEA that bundles an older milestone release. |
For a complete list of all closed issues and pull requests for this release, consult the 5.0 M6 milestone page in the JUnit repository on GitHub.
Java 9 Compatibility
The main runtime target of JUnit 5 is Java 8. Consequently, JUnit 5 artifacts do not ship
with compiled module descriptors for Java 9. However, since
milestone 5 each artifact published by JUnit 5 ships with a
stable Automatic-Module-Name declared in its JAR manifest. This allows test module
authors to require well-known JUnit module names as in the following example.
module foo.bar {
requires org.junit.jupiter.api;
}It is typically sufficient to run tests on the class path: in this regard there are no
changes between Java 8 and 9. All command-line tools and IDEs continue to work with JUnit
5 out of the box as long as they support the JUnit Platform. If your tool of choice
does not yet support the JUnit Platform, you may always resort to the ConsoleLauncher
or even to the executable junit-platform-console-standalone all-in-one jar.
Running JUnit Jupiter tests on the module path is implemented by pro, a Java 9 compatible build tool.
pro supports both black-box and white-box testing. The former is used by module surface
tests that only access the exported bits of the application modules. The latter uses a
merged module descriptor technique that allows access to protected and package-private
types as well as non-exported packages.
Consult the pro GitHub repository
for test module examples: integration.pro is a black-box test module; whereas,
com.github.forax.pro.api and com.github.forax.pro.helper are white-box test modules.
JUnit Platform
Bug Fixes
-
All tags are now trimmed in order to remove leading and trailing whitespace. This applies to any tag supplied directly to the
LauncherviaTagFilter.includeTags()andTagFilter.excludeTags()or indirectly via@IncludeTags,@ExcludeTags, the JUnit Platform Console Launcher, the JUnit Platform Gradle plugin, and the JUnit Platform Maven Surefire provider.
Deprecations and Breaking Changes
-
All tags must now conform to the following syntax rules:
-
A tag must not be
nullor blank. -
A trimmed tag must not contain whitespace.
-
A trimmed tag must not contain ISO control characters.
-
-
The
TagFilter.includeTags(),TagFilter.excludeTags(), andTestTag.create()factory methods now throw aPreconditionViolationExceptionif a supplied tag is not syntactically valid. -
The method
getDiscoveryFiltersByTypeofEngineDiscoveryRequesthas been renamed togetFiltersByType. -
The method
getSegments()ofUniqueIdnow returns an immutable list. -
The method
setSourceofAbstractTestDescriptorhas been removed. There is an additional constructor with asourceargument that can be used instead.
New Features and Improvements
-
New
TestTag.isValid(String)method for checking if a tag is syntactically valid. -
The
findAnnotation()method inAnnotationSupportnow searches on interfaces implemented by a class if the supplied element is a class. -
The following methods of
org.junit.platform.commons.util.ReflectionUtilsare now exposed viaorg.junit.platform.commons.support.ReflectionSupport:-
public static Optional<Class<?>> loadClass(String name) -
public static Optional<Method> findMethod(Class<?> clazz, String methodName, String parameterTypeNames) -
public static Optional<Method> findMethod(Class<?> clazz, String methodName, Class<?>… parameterTypes) -
public static <T> T newInstance(Class<T> clazz, Object… args) -
public static Object invokeMethod(Method method, Object target, Object… args) -
public static List<Class<?>> findNestedClasses(Class<?> clazz, Predicate<Class<?>> predicate)
-
JUnit Jupiter
Bug Fixes
-
All tags declared via
@Tagare now trimmed in order to remove leading and trailing whitespace. -
All primitive array types (from
boolean[]toshort[]) are now supported as a return type of static argument providing methods for parameterized tests.
Deprecations and Breaking Changes
-
A
voidreturn type is now enforced for@Testand lifecycle methods.
New Features and Improvements
-
All
fail(…)methods inAssertionscan now be used to implement single-statement lambda expressions, thereby avoiding the need to implement a code block with an explicit return value. -
New
getRoot()method inExtensionContextlets extension authors easily access the top-most, root extension context. -
New
getRequiredTestClass(),getRequiredTestInstance(), andgetRequiredTestMethod()convenience methods in theExtensionContextAPI that provide extension authors shortcuts for retrieving the test class, test instance, and test method in use cases for which such elements are required to be present. -
Type-level annotations such as
@TestInstanceand@Disabledmay now be declared on test interfaces (a.k.a., testing traits). -
A warning is now logged if more than one
TestDescriptoris resolved for a single method. This helps to debug errors resulting from a method being simultaneously annotated with multiple competing annotations such as@Test,@RepeatedTest,@ParameterizedTest,@TestFactory, etc. -
@Tagdeclarations containing invalid tag syntax will now be logged as a warning but effectively ignored.
5.0.0-RC1
Date of Release: July 30, 2017
Scope: Bug fixes and documentation improvements before 5.0 GA
| This is a pre-release and contains a few breaking changes. Please refer to the instructions above to use this version in a version of IntelliJ IDEA that bundles an older milestone release. |
For a complete list of all closed issues and pull requests for this release, consult the 5.0 RC1 milestone page in the JUnit repository on GitHub.
JUnit Platform
Bug Fixes
-
Generic, interface
defaultmethods that are not overridden by an implementing class can now be selected by class, method name, and parameter types or by fully qualified method name. This applies to method selectors inDiscoverySelectorsas well as to bothfindMethod()variants inReflectionSupport. -
A non-overridden interface default method whose method signature is overloaded by a locally declared method is now properly discovered when searching for methods within a class hierarchy using
findMethods()inReflectionSupport. -
Overridden interface default methods are no longer discovered when searching for methods within a class hierarchy using
findMethods()inReflectionSupport.
JUnit Jupiter
Bug Fixes
-
Configuration errors regarding lifecycle methods annotated with
@BeforeAll,@AfterAll,BeforeEach, or@AfterEachno longer halt the execution of the entire test plan during the discovery phase. Rather, such errors are now reported during the execution of the affected test class. -
A non-overridden interface default method whose method signature is overloaded by a locally declared method is now properly included in the test plan. This applies to default methods annotated with Jupiter annotations such as
@Test,@BeforeEach, etc. -
Overridden interface default methods are no longer included in the test plan. This applies to default methods annotated with Jupiter annotations such as
@Test,@BeforeEach, etc.
New Features and Improvements
-
New variants of
Assertions.assertThrows()that accept a custom failure message as aStringorSupplier<String>. -
Methods referenced by
@MethodSourceare no longer required to bestaticif the test class is annotated with@TestInstance(Lifecycle.PER_CLASS). -
Test classes written in the Kotlin programming language are now executed with
@TestInstance(Lifecycle.PER_CLASS)semantics by default.
5.0.0-RC2
Date of Release: July 30, 2017
Scope: Fix Gradle consumption of junit-jupiter-engine
| This is a pre-release and contains a few breaking changes. Please refer to the instructions above to use this version in a version of IntelliJ IDEA that bundles an older milestone release. |
For a complete list of all closed issues and pull requests for this release, consult the 5.0 RC2 milestone page in the JUnit repository on GitHub.
5.0.0-RC3
Date of Release: August 23, 2017
Scope: Configuration parameters and bug fixes.
| This is a pre-release and contains a few breaking changes. Please refer to the instructions above to use this version in a version of IntelliJ IDEA that bundles an older milestone or release candidate. |
For a complete list of all closed issues and pull requests for this release, consult the 5.0 RC3 milestone page in the JUnit repository on GitHub.
JUnit Platform
Bug Fixes
-
Source JARs no longer contain every source file twice.
-
The Maven Surefire provider now reports a failed test with a cause that is not an instance of
AssertionErroras an error instead of a failure for compatibility reasons.
New Features and Improvements
-
Configuration parameters can now be supplied in a number of new ways:
-
via a file named
junit-platform.propertiesin the root of the class path. See Configuration Parameters for details. -
via the
--configcommand-line option when using the Console Launcher. -
via the
configurationParameterorconfigurationParametersDSL when using the Gradle plugin. -
via the
configurationParametersproperty when using the Maven Surefire provider.
-
JUnit Jupiter
Bug Fixes
-
Source JARs no longer contain every source file twice.
-
ExecutionContext.Store.getOrComputeIfAbsent()now looks for values in a grandparent context (and recursively in its parents) before computing a value. -
ExecutionContext.Store.getOrComputeIfAbsent()is now thread safe. -
The
JupiterTestEngineno longer attempts to resolve a Unique ID selected via one of theDiscoverySelectors.selectUniqueId()methods if the Unique ID belongs to a different test engine.
Deprecations and Breaking Changes
-
Revert change introduced in RC1: test classes written in the Kotlin programming language are now executed with the same default test instance lifecycle mode as Java classes again (i.e., "per-method").
-
The
junit.conditions.deactivateconfiguration parameter has been renamed tojunit.jupiter.conditions.deactivate. -
The
junit.extensions.autodetection.enabledconfiguration parameter has been renamed tojunit.jupiter.extensions.autodetection.enabled. -
The default, global extension namespace constant in
ExtensionContexthas been renamed fromNamespace.DEFAULTtoNamespace.GLOBAL. -
The default
getStore()method has been removed from theExtensionContextinterface. To access the global store, use an explicit call togetStore(Namespace.GLOBAL)instead.
New Features and Improvements
-
The default test instance lifecycle mode can now be set via a configuration parameter or JVM system property named
junit.jupiter.testinstance.lifecycle.default. See Changing the Default Test Instance Lifecycle for details. -
When using
@CsvSourceor@CsvFileSourcein a parameterized test, if the CSV parser does not read any character from the input, and the input is within quotes, an empty string""is returned instead ofnull.
5.0.0
Date of Release: September 10, 2017
Scope: First General Availability Release
For a complete list of all closed issues and pull requests for this release, consult the 5.0 GA milestone page in the JUnit repository on GitHub.
JUnit Platform
Bug Fixes
-
The
removeFromHierarchy()implementation inAbstractTestDescriptornow also clears parent relationships from all children.
Deprecations and Breaking Changes
-
The
@APIannotation has been removed from thejunit-platform-commonsproject and relocated to a new standalone project on GitHub called the @API Guardian. -
Tags are no longer permitted to contain any of the following reserved characters.
-
,,(,),&,|,!
-
-
The constructor for
FilePositionhas been replaced with a static factory method namedfrom(int, int). -
A
FilePositioncan now be constructed solely from a line number via the newfrom(int)static factory method. -
FilePosition.getColumn()now returnsOptional<Integer>instead ofint. -
The constructors in the following concrete
TestSourceimplementations have been replaced with static factory methods namedfrom(…).-
ClasspathResourceSource -
ClassSource -
CompositeTestSource -
DirectorySource -
FileSource -
MethodSource -
PackageSource
-
-
The constructor for
LoggingListenerhas been replaced with a static factory method namedforBiConsumer(…). -
The
getParent()method inAbstractTestDescriptoris nowfinal.