Source code for this article can be found here.

Table of Contents

• Introduction
• Prerequisites
• Local Code Coverage
• Remote Code Coverage
• Conclusion

Introduction

Code coverage is an important metric that gives useful insight into what pieces of code your testing is executing or missing. According to their website, "JaCoCo is a free code coverage library for Java, which has been created by the EclEmma team based on the lessons learned from using and integration existing libraries for many years."

• JaCoCo instruments the resulting bytecode, not the source
• JaCoCo can pre-instrument the bytecode or run as a java agent, performing Just In Time instrumentation.
• JaCoCo is capable of instrumenting 1.7 bytecode

Like all code coverage tools, JaCoCo is not capable of reporting how "good" your tests are, only that code is being exercised in some manner.

Prerequisites

• Java
• JaCoCo
• TestNG
• Ant (Optional)
• Gradle (Optional)
• cURL (Optional)

It is important to have a standardized build process defined for your project (and for the sanity of your team), especially when you get to the point where you are considering adding in-depth topics such as code coverage. However, JaCoCo can be run independent of any particular build system. In this article I will demonstrate usage of JaCoCo with both Gradle (for local coverage) and Ant (for remote coverage).

Local Code Coverage

When using Gradle, running JaCoCo is as simple as listing the jacoco Gradle plugin. Here is the build file for our jacoco-demo project.

build.gradle

apply plugin: 'java'
apply plugin: 'war'
apply plugin: 'eclipse'
apply plugin: 'eclipse-wtp'
apply plugin: 'jacoco'

defaultTasks 'build'

group = "com.toastedbits.jacoco"
version = "1.0-SNAPSHOT"

sourceCompatibility = '1.7'
targetCompatibility = '1.7'

buildDir = 'target'

repositories {
    mavenCentral()
}

dependencies {
    compile 'org.testng:testng:6.8.5'
	providedCompile 'org.apache.tomcat:servlet-api:6.0.37'
}

test {
    useTestNG() {
        suiteXmlBuilder().suite(name: 'jacoco demo test suite', parallel: 'tests') {
            test (name: 'all-tests') {
                packages {
                    'package' (name:'com.toastedbits.jacoco.test.*')
                }
            }
        }
    }
    ext.useDefaultListeners = true
}

eclipse {
    wtp {
        //By default gradle excludes provided dependencies with the default value for minusConfigurations,
        //We want these dependencies to appear in our eclipse project since we dont run eclipse from within the tomcat environment
        component {
            plusConfigurations += configurations.providedRuntime
            minusConfigurations = null
        }
        //The default facet configuration generated by the eclipse-wtp plugin defaults to outdated values
        //This causes unusual phony errors to be reported by eclipse when using jdk 1.7 features, let's bring them up to date
        facet {
            facet name: 'jst.web', version: '3.0'
            facet name: 'java', version: '1.7'
            facet name: 'wst.jsdt.web', version: '1.0'
        }
    }
}

Here since the testing is so small, we are inlining creation of the TestNG suite. This is not generally recommended, especially for larger projects. It is also best practice to create a settings.gradle so the project name doesnt change on us based on our folder name. This file also allows us to introduce multi-project build concepts into larger projects.

settings.gradle

rootProject.name = 'jacoco-demo'

The demo project is laid out according to standard java builds used by Gradle (and Maven).

build.gradle
settings.gradle
src/main/java/com/toastedbits/jacoco/
	JaCoCoDemo.java
	DemoServlet.java
src/main/webapp/WEB-INF/
	web.xml
src/test/java
	DemoTest.java

For now we will ignore the webapp related code (servlet and web.xml) and focus only on the DemoTest and JaCoCoDemo classes.

DemoTest.java

package com.toastedbits.jacoco.test;

import static org.testng.Assert.*;
import java.sql.SQLException;
import org.testng.annotations.Test;
import com.toastedbits.jacoco.JaCoCoDemo;

public class DemoTest {
	@Test
	public void testGetConnection() throws SQLException {
		JaCoCoDemo jccd = new JaCoCoDemo();
		assertEquals(42, jccd.calculateAnswer("What is the answer to the ultimate question of life, the universe and everything?"));
	}
}

JaCoCoDemo.java

package com.toastedbits.jacoco;

public class JaCoCoDemo {
	private static final int ANSWER = 42;
	private static final int WRONG_ANSWER = 314159;
	
	public int calculateAnswer(String question) {
		if(question.contains("life")) {
			return ANSWER;
		}
		else {
			return WRONG_ANSWER;
		}
	}
}

Not too much going on here, just unit testing a simple class named JaCoCoDemo that contains a method returning 42 if the question contains the string "life". I left out the negative test so we get a more interesting picture when running JaCoCo.

We can execute our local tests and see the JaCoCo report by executing gradle jacoco in the root project folder. JaCoCo reports can then be found in target/reports/jacoco

See Gradle's documentation on the JaCoCo plugin for more information

Remote Code Coverage

Now that we have a simple understanding of what JaCoCo can do for us, lets remove some of the magic that Gradle performs for us and move into Ant so we can see what really goes on behind the scenes. I have created a simple Ant build.xml that builds the war file and generates coverage reports generated from integration tests against a basic servlet that makes use of the same JaCoCoDemo class. Here we will stub our integration tests with cURL, but your project may wish to integrate more with TestNG by using Apache HttpComponents, Selenium, etc.

First we need to ensure our container environment has the jacocoagent.jar on its classpath and is used as a java agent. For Tomcat, this means we must add jacocoagent.jar to $TOMCAT_HOME/lib and ensure Tomcat's execution environment has CATALINA_OPTS set to use the agent:

CATALINA_OPTS="$CATALINA_OPTS -javaagent:$TOMCAT_HOME/lib/jacocoagent.jar=append=false,dumponexit=false,output=tcpserver,address=*,port=6300,excludes=com.example.excludeme*"

Some useful tips:

• This can be added to the top of $TOMCAT_HOME/bin/catalina.sh to enforce jacoco to always run.
• Also a good oportunity to allow access to jvisualvm (java profiler) with JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.port=8086 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
• Remember to open port 6300, disable SELinux, etc on the host running Tomcat

Here are the steps needed to generate a jacoco report for a remote host:

1. Build the war file with ant or gradle (default targets build the war)
2. execute curl -k -v "http://example.com/jacoco-demo/?q=What%20is%20the%20meaning%20of%20life%20the%20universe%20and%20everything"
3. execute: ant jacocoReport
4. open: target/report/index.html

The webapp itself is very basic

web.xml

<web-app xmlns="http://java.sun.com/xml/ns/javaee"
		 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
		 xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
		 version="3.0">
	
	<servlet>
		<servlet-name>DemoServlet</servlet-name>
		<servlet-class>com.toastedbits.jacoco.DemoServlet</servlet-class>
	</servlet>

	<servlet-mapping>
		<servlet-name>DemoServlet</servlet-name>
		<url-pattern>/</url-pattern>
	</servlet-mapping>
</web-app>

DemoServlet.java

package com.toastedbits.jacoco;

import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class DemoServlet extends HttpServlet {
	private static final long serialVersionUID = -7485296299041069908L;
	
	@Override
	public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException {
		String question = request.getParameter("q");
		
		PrintWriter pw = response.getWriter();
		pw.print("<!DOCTYPE html><html><head><meta charset=\"utf-8\"><title>Answer</title></head><body>");
		
		if(question != null) {
			pw.print("<p>Question: " + question + "<br>");
			pw.print("Answer: " + new JaCoCoDemo().calculateAnswer(question) + "</p>");
		}
		else {
			pw.print("<p>Pose a question with the q request parameter</p>");
		}
		
		pw.print("</body></html>");
		
		pw.flush();
	}
}

build.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE project>
<project name="jacoco-demo" basedir="." default="build" xmlns:jacoco="antlib:org.jacoco.ant">
	<property name="sourceCompatibility" value="1.7"/>
	<property name="targetCompatibility" value="1.7"/>
	
	<property name="target" location="${basedir}/target"/>
	<property name="target.classes" location="${target}/classes"/>
	<property name="target.classes.main" location="${target.classes}/main"/>
	<property name="src.main" location="${basedir}/src/main"/>
	<property name="src.main.java" location="${src.main}/java"/>
	<property name="src.main.webapp" location="src/main/webapp"/>
	<property name="webinf.dir" location="${src.main.webapp}/WEB-INF"/>
	<property name="metainf.dir" location="${src.main.webapp}/META-INF"/>
	
	<property name="war.file.name" value="jacoco-demo.war"/>
	
	<property name="jacoco.host" value="rcdn6-vm63-54"/>
	<property name="jacoco.port" value="6300"/>
	<property name="jacoco.datafile" value="${coverage.dir}/jacoco.exec"/>
	
	<path id="compile.classpath">
	</path>
	
	<path id="providedCompile.classpath">
		<fileset dir="${basedir}/lib" includes="servlet-api*.jar"/>
	</path>
	
	<target name="build" depends="war"/>
	
	<target name="war" depends="compile">
		<war destfile="${target}/${war.file.name}" webxml="${webinf.dir}/web.xml" compress="false">
			<metainf dir="${metainf.dir}"/>
			 <!-- The webinf task is a little dumb, it thinks classes and libs should be in with our source tree and wants to re-include the web.xml, exclude them for better control -->
			<webinf dir="${webinf.dir}">
				<exclude name="web.xml"/>
				<exclude name="classes/**"/>
				<exclude name="lib/**"/>
			</webinf>
			<classes dir="${target.classes.main}"/>
			<lib dir="${webinf.dir}/lib"/>
			<fileset dir="${src.main.webapp}">
				<exclude name="WEB-INF/**"/>
				<exclude name="META-INF/**"/>
			</fileset>
		</war>
	</target>
	
	<target name="compile">
		<mkdir dir="${target.classes.main}"/>
		<javac
			debug="true"
			srcdir="${src.main.java}"
			destdir="${target.classes.main}"
			source="${sourceCompatibility}"
			target="${targetCompatibility}"
			includeantruntime="false"
			fork="true">
			
			<classpath refid="compile.classpath"/>
			<classpath refid="providedCompile.classpath"/>
		</javac>
	</target>
	
    <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
		<classpath path="${basedir}/lib/jacocoant.jar"/>
	</taskdef>
	
	<target name="jacocoReport">
		<delete file="${jacoco.datafile}" />
		<jacoco:dump address="${jacoco.host}" port="${jacoco.port}" dump="true" reset="true" destfile="${target}/remote.exec" append="false"/>
		<jacoco:report>
			<executiondata>
				<file file="${target}/remote.exec"/>
			</executiondata>
			<structure name="Coverage">
				<classfiles>
					<fileset dir="${target}/classes"/>
				</classfiles>
				<sourcefiles encoding="UTF-8">
					<fileset dir="src"/>
				</sourcefiles>
			</structure>
			<html destdir="${target}/report"/>
			<xml destfile="${target}/report/report.xml"/>
		</jacoco:report>
	</target>
	
	<target name="clean">
		<delete dir="${target}"/>
	</target>
</project>

Troubleshooting: Jacoco dynamically instruments classes with private data. If tomcat fails to start or you encounter other bugs with jacoco reporting, check for code that performs unsafe reflection (iterating through all members of a class without validating access level) may need to be excluded or rewritten.

Conclusion

Code coverage gives important insight into the testing of your code, but this power is even greater when included as part of your daily workflow. JaCoCo reporting doesn't have to be a manual process. I recommend adding it as part of a larger automated workflow with Git, Gerrit, Jenkins, and Sonar. The best part is all of this is free and open source! Setting up a workflow for your team may cost a few days up front, but the gains far outweigh any initial time investment.