JUnit 5 TestWatcher API

1. Overview

When unit testing, we may, periodically, wish to process the results of our test method executions. In this quick tutorial, we’ll take a look at how we can accomplish this using the TestWatcher API provided by JUnit.

For an in-depth guide to testing with JUnit, check out our excellent Guide to JUnit 5.

2. The TestWatcher API

In short, the TestWatcher interface defines the API for extensions that wish to process test results. One way we can think of this API is providing hooks for getting the status of an individual test case.

But, before we dive into some real examples, let’s take a step back and briefly summarize the methods in the TestWatcher interface:

• testAborted​(ExtensionContext context, Throwable cause)

  To process the results of an aborted test, we can override the testAborted method. As the name suggests, this method is invoked after a test has been aborted.

• testDisabled​(ExtensionContext context, Optional reason)

  We can override the testDisabled method when we want to handle the results of a disabled test method. This method may also include the reason the test is disabled.

• testFailed(ExtensionContext context, Throwable cause)

  If we want to do some additional processing after a test failure, we can simply implement the functionality in the testFailed method. This method may include the cause of the test failure.

• testSuccessful(ExtensionContext context)

  Last but not least, when we wish to process the results of a successful test, we simply override the testSuccessful method.

We should note that all the methods contain the ExtensionContext. This encapsulates the context in which the current test executed.

3. Maven Dependencies

First of all, let’s add the project dependencies we will need for our examples.
Apart from the main JUnit 5 library junit-jupiter-engine, we’ll also need the junit-jupiter-api library:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <version>5.4.2</version>
    <scope>test</scope>
</dependency>

As always, we can get the latest version from Maven Central.

4. A TestResultLoggerExtension Example

Now that we have a basic understanding of the TestWatcher API, we’ll walk through a practical example.

Let’s begin by creating a simple extension for logging the results and providing a summary of our tests. In this case, to create the extension, we need to define a class that implements the TestWatcher interface:

public class TestResultLoggerExtension implements TestWatcher, AfterAllCallback {
    private List<TestResultStatus> testResultsStatus = new ArrayList<>();

    private enum TestResultStatus {
        SUCCESSFUL, ABORTED, FAILED, DISABLED;
    }

    //...
}

As with all extension interfaces, the TestWatcher interface also extends the main Extension interface, which is only a marker interface. In this example, we also implement the AfterAllCallback interface.

In our extension, we have a list of TestResultStatus, which is a simple enumeration we’re going to use to represent the status of a test result.

4.1. Processing the Test Results

Now, let’s see how to process the results of the individual unit test method:

@Override
public void testDisabled(ExtensionContext context, Optional<String> reason) {
    LOG.info("Test Disabled for test {}: with reason :- {}", 
      context.getDisplayName(),
      reason.orElse("No reason"));

    testResultsStatus.add(TestResultStatus.DISABLED);
}

@Override
public void testSuccessful(ExtensionContext context) {
    LOG.info("Test Successful for test {}: ", context.getDisplayName());

    testResultsStatus.add(TestResultStatus.SUCCESSFUL);
}  

We begin by filling the body of our extension and overriding the testDisabled() and testSuccessful() methods.

In our trivial example, we output the name of the test and add the status of the test to the testResultsStatus list.

We’ll continue in this fashion for the other two methods — testAborted() and testFailed():

@Override
public void testAborted(ExtensionContext context, Throwable cause) {
    LOG.info("Test Aborted for test {}: ", context.getDisplayName());

    testResultsStatus.add(TestResultStatus.ABORTED);
}

@Override
public void testFailed(ExtensionContext context, Throwable cause) {
    LOG.info("Test Aborted for test {}: ", context.getDisplayName());

    testResultsStatus.add(TestResultStatus.FAILED);
}

4.2. Summarizing the Test Results

In the last part of our example, we’ll override the afterAll() method:

@Override
public void afterAll(ExtensionContext context) throws Exception {
    Map<TestResultStatus, Long> summary = testResultsStatus.stream()
      .collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));

    LOG.info("Test result summary for {} {}", context.getDisplayName(), summary.toString());
}

To quickly recap, the afterAll method is executed after all test methods have been run. We use this method to group the different TestResultStatus we have in the list of test results before outputting a very basic summary.

For an in-depth guide to Lifecycle Callbacks, check out our excellent Guide to JUnit 5 extensions.

5. Running the Tests

In this penultimate section, we’ll see what the output from our tests looks like using our simple logging extension.

Now that we’ve defined our extension, we’ll first register it using the standard @ExtendWith annotation:

@ExtendWith(TestResultLoggerExtension.class)
class TestWatcherAPIUnitTest {

    @Test
    void givenFalseIsTrue_whenTestAbortedThenCaptureResult() {
        Assumptions.assumeTrue(false);
    }

    @Disabled
    @Test
    void givenTrueIsTrue_whenTestDisabledThenCaptureResult() {
        Assert.assertTrue(true);
    }

    //...

Next, we fill our test class with unit tests, adding a mixture of disabled, aborted, and successful tests.

5.1 Reviewing the Output

When we run the unit test, we should see the output for each test:

INFO  c.b.e.t.TestResultLoggerExtension - 
    Test Successful for test givenTrueIsTrue_whenTestAbortedThenCaptureResult()
...
Test result summary for TestWatcherAPIUnitTest {ABORTED=1, SUCCESSFUL=1, DISABLED=2}

Naturally, we’ll also see the summary printed when all the test methods have completed.

6. Gotchas

In this last section, let’s review a couple of the subtleties we should be aware of when working with the TestWatcher interface:

• TestWatcher extensions are not permitted to influence the execution of tests; this means if an exception is thrown from a TestWatcher, it will not be propagated up to the running test
• Currently, this API is only used to report the results of @Test methods and @TestTemplate methods
• By default, if no reason is provided to the testDisabled method, then it will contain the fully qualified name of the test method followed by ‘is @Disabled‘

7. Conclusion

To summarize, in this tutorial, we’ve shown how we can make use of the JUnit 5 TestWatcher API to process the results of our test method executions.

The full source code of the examples can be found over on GitHub.