A Guide to JaCoCo

1. Overview

Code coverage is a software metric used to measure how many lines of our code are executed during automated tests.

In this article we’re going to stroll through some practical aspects of using JaCoCo – a code coverage reports generator for Java projects.

2. Maven Configuration

In order to get up and running with JaCoCo, we need to declare this maven plugin in our pom.xml file:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.7.7.201606060606</version>
    <executions>
        <execution>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>report</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The link provided here-before will always lead you to the latest version of the plugin in the maven central repository.

3. Code Coverage Reports

Before we start looking at JaCoCo’s code coverage capabilities, we need to have a code sample. Here’s a simple Java function that checks whether a string reads the same backward and forward:

public boolean isPalindrome(String inputString) {
    if (inputString.length() == 0) {
        return true;
    } else {
        char firstChar = inputString.charAt(0);
        char lastChar = inputString.charAt(inputString.length() - 1);
        String mid = inputString.substring(1, inputString.length() - 1);
        return (firstChar == lastChar) && isPalindrome(mid);
    }
}

All we need now is a simple JUnit test:

@Test
public void whenEmptyString_thenAccept() {
    Palindrome palindromeTester = new Palindrome();
    assertTrue(palindromeTester.isPalindrome(""));
}

Running the test using JUnit will automatically set in motion the JaCoCo agent, thus, it will create a coverage report in binary format in the target directory – target/jacoco.exec.

Obviously we cannot interpret the output single-handedly, but other tools and plugins can – e.g. Sonar Qube.

The good news is that we can use the jacoco:report goal in order to generate readable code coverage reports in several formats – e.g. HTML, CSV, and XML.

We can now take a look for example at target/site/jacoco/index.html page to see what the generated report looks like:

Following the link provided in the report – Palindrome.java , we can drill through a more detailed view for each Java class:

Note that you can straightforwardly manage code coverage using JaCoCo inside Eclipse with zero configuration, thanks to EclEmma Eclipse plugin.

4. Report Analysis

Our report shows 21% instructions coverage, 17% branches coverage, 3/5 for cyclomatic complexity and so on.

The 38 instructions shown by JaCoCo in the report refers to the bytecode instructions as opposed to ordinary Java code instructions.

JaCoCo reports help to visually analyze code coverage by using diamonds colors for branches and background colors for lines:

• Red diamond means that no branches have been exercised during the test phase.
• Yellow diamond shows that the code is partially covered – some branches have not been exercised.
• Green diamond means that all branches have been exercised during the test.

The same color code applies on the background color, but for lines coverage.

JaCoCo mainly provide three important metrics:

• Lines coverage reflects the amount of code that has been exercised based on the number of Java byte code instructions called by the tests.
• Branches coverage shows the percent of exercised branches in the code – typically related to if/else and switch statements.
• Cyclomatic complexity reflects the complexity of code by giving the number of paths needed to cover all the possible paths in a code through linear combination.

To take a trivial example, if there is no if or switch statements in the code, the cyclomatic complexity will be 1, as we only need one execution path to cover the entire code.

Generally the cyclomatic complexity reflects the number of test cases we need to implement in order to cover the entire code.

5. Concept Breakdown

JaCoCo runs as a Java agent, it is responsible for instrumenting the bytecode while running the tests. JaCoCo drills into each instruction and shows which lines are exercised during each test.

To gather coverage data, JaCoCo uses ASM for code instrumentation on the fly, receiving events from the JVM Tool Interface in the process:

It is also possible to run the JaCoCo agent is server mode, in this case, we can run our tests with jacoco:dump as a goal, to initiate a dump request.

You can follow the official documentation link for more in-depth details about JaCoCo design.

6. Code Coverage Score

Now that we know a bit about how JaCoCo works, let’s improve our code coverage score.

In order to achieve 100% code coverage we need to introduce tests, that covers the missing parts shown in the initial report:

@Test
public void whenPalindrom_thenAccept() {
    Palindrome palindromeTester = new Palindrome();
    assertTrue(palindromeTester.isPalindrome("noon"));
}
    
@Test
public void whenNotPalindrom_thenReject(){
    Palindrome palindromeTester = new Palindrome();
    assertFalse(palindromeTester.isPalindrome("neon"));
}

Now we can say that we have enough tests to cover our the entire code, but to make sure of that, let’s run the Maven command mvn jacoco:report to publish the coverage report:

As you can see all lines/branches/paths in our code are fully covered:

In real world project, and as developments go further, we need to keep in track the code coverage score.

JaCoCo offers a simple way of declaring minimum requirements that should be met, otherwise the build will fail.

We can do that by adding the following check goal in our pom.xml file:

<execution>
    <id>jacoco-check</id>
    <goals>
        <goal>check</goal>
    </goals>
    <configuration>
        <rules>
            <rule>
                <element>PACKAGE</element>
                <limits>
                    <limit>
                        <counter>LINE</counter>
                        <value>COVEREDRATIO</value>
                        <minimum>0.50</minimum>
                    </limit>
                </limits>
            </rule>
        </rules>
    </configuration>
</execution>

As you can probably guess, we’re limiting here the minimum score for lines coverage to 50%.

The jacoco:check goal is bound to verify, so we can run the Maven command – mvn clean verify to check whether the rules are respected or not. The logs will show something like:

[ERROR] Failed to execute goal org.jacoco:jacoco-maven-plugin:0.7.7.201606060606:check 
  (jacoco-check) on project mutation-testing: Coverage checks have not been met.

7. Conclusion

In this article we’ve seen how to make use of JaCoCo maven plugin to generate code coverage reports for Java projects.

Keep in mind though, 100% code coverage does not necessary reflects effective testing, as it only reflects the amount of code exercised during tests. In a previous article, we’ve talked about mutation testing as a more sophisticated way to track tests effectiveness compared to ordinary code coverage.

You can check out the example provided in this article in the linked GitHub project.