This document contains the change log for all JUnit 5 releases since 5.2 GA.

Please refer to the User Guide for comprehensive reference documentation for programmers writing tests, extension authors, and engine authors as well as build tool and IDE vendors.

5.3.2

Date of Release: November 25, 2018

Scope: Bug fixes since 5.3.1

For a complete list of all closed issues and pull requests for this release, consult the 5.3.2 milestone page in the JUnit repository on GitHub.

JUnit Platform

Bug Fixes

  • When configured with --details verbose, the ConsoleLauncher no longer throws a MissingFormatArgumentException if a test method display name contains String format specifiers such as %c.

JUnit Jupiter

Bug Fixes

  • Assertions.assertAll() is now thread-safe — for example, it can now be used with a parallel Stream.

  • The OS.SOLARIS enum constant used with @EnabledOnOs and @DisabledOnOs is now also detected as the current operating system if the os.name JVM system property contains "SunOs".

  • Assertions.assertLinesMatch() no longer throws a NullPointerException after evaluating a fast-forward match if there are more expected lines after the fast-forward match than remain in the actual results. This bug only manifested itself if the expected list size was equal to or greater than the actual list size.

  • Multidimensional arrays may now be supplied to @ParameterizedTest methods from factory methods configured via @MethodSource.

    • For example, a factory method with the signature static Stream<int[][]> factory() can be used as the @MethodSource for a @ParameterizedTest with the signature void test(int[][]).

  • Threads created for running tests in parallel now use the same thread context class loader (TCCL) that was set when creating the underlying executor service. This resolves ClassNotFoundException issues that only occur in parallel execution mode when a custom TCCL is in place.

  • When executing tests in parallel, lifecycle methods and callbacks called after a @TestFactory method are now always executed after the dynamic tests returned by the method.

  • Exceptions thrown during initialization of static @RegisterExtension fields now cause the test class to fail instead of being silently swallowed.

JUnit Vintage

No changes

5.3.1

Date of Release: September 11, 2018

Scope: Bug fixes since 5.3.0

For a complete list of all closed issues and pull requests for this release, consult the 5.3.1 milestone page in the JUnit repository on GitHub.

JUnit Platform

Bug Fixes

  • An OutOfMemoryError regression introduced in JUnit 5.3.0 has been fixed.

    • Specifically, the NodeTestTask used by implementations of HierarchicalTestEngine (such as the Jupiter and Vintage test engines) no longer retains references to contextual state after a node has completed execution. This allows state such as instances of test classes to be properly garbage collected by the JVM.

    • Previously, a NodeTestTask instance was created for each TestDescriptor before starting execution. Now they are created on the fly and can be garbage collected by the JVM after the enclosing container has finished.

  • The OpenTest4J dependency has been updated to 1.1.1 to fix a serialization incompatibility between 1.0.0 and 1.1.0 that caused failure messages to be discarded when used from Gradle and potentially other tools and IDEs.

JUnit Jupiter

Bug Fixes

  • Invocations of assertThrows() that are passed a method reference for an overloaded method with a void return type once again compile.

    • For example, given an instance of java.lang.Object stored in a variable named object, assertThrows(Exception.class, object::wait) compiled against JUnit 5.2.0, failed to compile against JUnit 5.3.0, but now compiles against JUnit 5.3.1.

Breaking Changes

  • In order to revert the aforementioned breaking change, variants of assertThrows() introduced in JUnit 5.3.0 that accept ThrowingSupplier arguments have been removed.

JUnit Vintage

No changes

5.3.0

Date of Release: September 3, 2018

Scope: Parallel test execution, output capture for System.out and System.err, new TestInstanceFactory extension API, custom test sources for dynamic tests, promotion of the dynamic test API from experimental to maintained status, discontinuation of the junit-platform-gradle-plugin, deprecation of the junit-platform-surefire-provider, as well as various minor improvements and bug fixes.

For a complete list of all closed issues and pull requests for this release, consult the 5.3 M1, 5.3 RC1, and 5.3 GA milestone pages in the JUnit repository on GitHub.

JUnit Platform

Bug Fixes

  • The full stacktrace is now printed to the console when running the ConsoleLauncher in --details verbose mode.

  • ReflectionUtils.findNestedClasses() and ReflectionSupport.findNestedClasses() no longer allow a NoClassDefFoundError to propagate if a nested class or nested interface has an invalid class file. Instead, the error will now be swallowed and logged at WARNING level.

  • All DiscoverySelector implementations (e.g., PackageSelector, ClassSelector, MethodSelector, etc.) now implement equals() and hashCode() for proper behavior when stored in collections.

  • ClassSource has been revised so that equals() and hashCode() are now properly based on the required class name instead of the optional Class reference. In addition, the non-blank precondition for a class name is now enforced.

Deprecations and Breaking Changes

  • The junit-platform-gradle-plugin has been discontinued and is no longer released as part of JUnit 5. Please use Gradle’s native support for running tests on the JUnit Platform (requires Gradle 4.6 or higher) instead.

  • The JUnit Platform Surefire Provider (junit-platform-surefire-provider) is now deprecated in favor of the native support for the JUnit Platform provided by Maven Surefire 2.22.0 and later versions.

  • The JUnit Platform Launcher now enforces that only TestEngine implementations published by the JUnit Team may use the junit- prefix for their TestEngine IDs.

  • The findAnnotation() methods in AnnotationSupport and AnnotationUtils no longer cache annotation lookups. Note, however, that the algorithm remains otherwise unmodified and is therefore semantically identical to the previous behavior.

New Features and Improvements

  • Reusable support for parallel test execution for test engines that extend HierarchicalTestEngine.

    • HierarchicalTestEngine implementations may now specify a HierarchicalTestExecutorService.

    • By default, a SameThreadHierarchicalTestExecutorService is used.

    • Test engines may use ForkJoinPoolHierarchicalTestExecutorService to support parallel test execution based on Java’s Fork/Join framework.

    • Node implementations may provide a set of ExclusiveResources and an ExecutionMode to be used by ForkJoinPoolHierarchicalTestExecutorService.

  • Experimental support for capturing output printed to System.out and System.err during test execution.

    • This feature is disabled by default but can be enabled using a configuration parameter (see the User Guide for details).

    • If the new experimental feature is enabled, output captured for System.out and System.err will be written to the dedicated system-out and system-err elements, respectively, in the XML report generated by the ConsoleLauncher.

  • New UriSource.from(URI) static factory method that allows a TestSource to be created from a URI. If the URI references a file or directory in the local filesystem, a FileSource or DirectorySource will be created; otherwise, an instance of the default UriSource implementation will be created.

  • New MethodSource.from(Class, Method) static factory method for creating a MethodSource from a specific class and method. This method should be used in favor of MethodSource.from(Method) when the test method is inherited from a superclass or present as an interface default method.

  • A ClasspathResourceSource can now be created from a URI via the new from(URI) static factory method if the URI uses the classpath scheme.

  • New overloaded variant of isAnnotated() in AnnotationSupport that accepts Optional<? extends AnnotatedElement> instead of AnnotatedElement.

  • New LauncherConfig and associated builder for configuring the LauncherFactory. Specifically, auto-registration of test engines and test execution listeners can now be disabled, and additional engines and listeners can be registered programmatically.

  • New --fail-if-no-tests command-line option for the ConsoleLauncher.

    • When this option is enabled and no tests are discovered, the launcher will fail and exit with a status code of 2.

  • The ConsoleLauncher now uses the picocli library under the hood to parse the command line and generate usage help.

    • Users may now pass command-line arguments via an argument file (@-file) to the console launcher. Argument files allow users to work around system limitations on the length of a command line when creating a command line with lots of options or with long arguments for options.

    • The usage help is now displayed using ANSI colors on supported platforms.

JUnit Jupiter

Bug Fixes

  • When using @TestInstance(Lifecycle.PER_CLASS) semantics, registered AfterAllCallback extensions are no longer invoked if an exception is thrown by the test class constructor. Consequently, AfterAllCallback extensions are now only invoked if BeforeAllCallback extensions are invoked.

  • Exceptions thrown in @After and @AfterAll lifecycle methods now take precedence over violated assumptions (i.e. TestAbortedExceptions) in test or prior lifecycle methods.

  • The MethodSource for an inherited @Test method now correctly references the current test class instead of the class or interface in which the @Test method is declared. This allows build tools such as Maven Surefire to properly include inherited @Test methods when executing a single test class or specific test classes based on filters — for example, via mvn test -Dtest=SubclassTests).

  • Test discovery no longer halts prematurely if a nested class or nested interface in a test class has an invalid class file.

  • Certain categories of errors encountered during the test discovery phase no longer cause JUnit Jupiter to prematurely abort the entire discovery process.

    • Such errors are now logged, thereby enabling JUnit Jupiter to discover and execute as many tests as possible while still informing the user of containers and tests that could not be properly discovered.

Bug Fixes since 5.3 RC1

  • The junit-jupiter-params JAR file was missing the required Kotlin metadata for the getAs<Class>(index) Kotlin extension method. The underlying packaging issue has been solved, and the extension method has been renamed to get<Class>(index) as originally intended in 5.3.0-M1.

  • TestInstanceFactory extensions are properly inherited within @Nested test class hierarchies.

Breaking Changes since 5.3 RC1

  • @Nested test classes can no longer override TestInstanceFactory extensions registered on an enclosing class. This aligns with the behavior for TestInstanceFactory extensions registered within a conventional test class hierarchy.

New Features and Improvements

  • Experimental support for parallel test execution. By default, tests are still executed sequentially, but parallelism can be enabled using a configuration parameter (please refer to the User Guide for examples and configuration options).

  • New assertThrows methods in Assertions provide a more specific failure message if the supplied lambda expression or method reference returns a result instead of throwing an exception.

  • Generation of a detailed failure message for a failed assertion no longer fails if the toString() implementation of an object supplied to the assertion throws an exception. Instead, the object with the broken toString() implementation will be referenced via a default String representation based on the object’s fully qualified class name and system hash code, separated by an @ symbol.

  • Although it is highly discouraged, it is now possible to extend the org.junit.jupiter.api.Assertions and org.junit.jupiter.api.Assumptions classes for special use cases.

  • New publishEntry(String) method in TestReporter that makes it easier to publish a report entry based solely on a value without requiring that a key be specified (as is required by the existing publishEntry() variants).

  • New support for the IBM AIX operating system in @EnabledOnOs and @DisabledOnOs.

  • The dynamic test API has been promoted from experimental to maintained status. This affects the @TestFactory annotation as well as the DynamicTest, DynamicContainer, and DynamicNode types in the org.junit.jupiter.api package.

  • New support for supplying a custom test source URI when creating a dynamic container or test.

    • A custom test source URI for a dynamic container or dynamic test will be registered as a ClasspathResourceSource if the URI uses the classpath scheme; otherwise, such a URI will be registered as a FileSource, DirectorySource, or UriSource as appropriate.

    • See the new factory methods dynamicContainer(String, URI, …​) in DynamicContainer and dynamicTest(String, URI, Executable) in DynamicTest for details.

  • New {displayName} placeholder for the name attribute in @ParameterizedTest that allows developers to include the display name of the @ParameterizedTest method in a custom display name for invocations of that parameterized test.

    • This aligns with the existing {displayName} placeholder support for @RepeatedTest.

  • Generation of the display name for a @ParameterizedTest no longer fails if the toString() implementation of an argument for the parameterized test throws an exception. Instead, the object with the broken toString() implementation will be referenced via a default String representation based on the object’s fully qualified class name and system hash code, separated by an @ symbol.

  • Implicit argument conversion for parameterized tests can now convert strings such as "java.lang.Integer", "long", and "byte[]" to Class instances.

  • New arguments() static factory method in the Arguments interface that serves as an alias for Arguments.of(). arguments() is intended to be used via import static.

  • New get<Class>(index) Kotlin extension method to make ArgumentsAccessor friendlier to use from Kotlin.

  • ArgumentConverters and ArgumentsAggregators registered using @ConvertWith and @AggregateWith, respectively, are now only instantiated once per @ParameterizedTest instead of once for each invocation.

  • Performance improvements for executing parameterized tests, particularly when the method declares more than a few parameters.

  • New TestInstanceFactory extension API that enables custom creation of test class instances.

JUnit Vintage

Bug Fixes

  • The MethodSource for an inherited @Test method now correctly references the current test class instead of the class or interface in which the @Test method is declared. This allows build tools such as Maven Surefire to properly include inherited @Test methods when executing a single test class or specific test classes based on filters — for example, via mvn test -Dtest=SubclassTests).

New Features and Improvements

  • The VintageTestEngine now uses the simple name of a test class as the display name instead of the fully qualified class name. This aligns with the behavior of the JupiterTestEngine.

5.2.0

Date of Release: April 29, 2018

Scope: JUnit BOM, support for Maven Surefire 2.21.0 allowing builds with Java 9 and Java 10, argument aggregation and widening primitive conversion for arguments in parameterized tests, external factory methods for @MethodSource, as well as various minor improvements and bug fixes.

For complete details consult the 5.2.0 Release Notes online.