At my work place we've been striving for a much larger base of tests for our code. Not satisfied with simply estimating how much of our code is covered by our tests, we decided to take it a step further and get actual numbers with a free and open source tool, Cobertura. The documentation is decent, but leaves bits for you to 'fill in the gap' conceptually, this is especially apparent when trying to run Cobertura against a webapp or other externally managed environment outside of an application with a simple main routine.
You will first need a copy of Cobertura from their download page. Keep your copy of cobertura in a centralized location. Depending on how often you wish to use this tool you may consider a variety of instrumentation strategies.
Cobertura integrates easily with Ant and Maven, our team's project is Ant based (migration to something newer is on the roadmap!) or can be run manually with the script files bundled. The ant tasks/Maven plugins work much the same way as running them manually and documentation for each is easily found on the web (Ant | Maven). Since manually running the tools gives the best insight into all the inner workings, that's what I'll be demonstrating with here!
Pretty much says it all, in order for cobertura to work you need to compile your project and have the class files available. One thing to note, cobertura doesn't care about any bundled artificats at this point (.jar, .war, etc). We are only interested in the class files at this point!
After building your project, we need to run cobertura-instrument on the built .class files.
Assuming your class files are output to a folder called build run: cobertura-instrument.sh --basedir build
This will perform an inline replacement of your class files with the instrumented versions. Ant and Maven both provide hooks to perform this step
In addition to the instrumented classes, a file called cobertura.ser will be generated. Copy this out to the location you specify for ""net.sourceforge.cobertura.datafile" in the next step.
Cobertura injects itself as a runtime only dependency. Executing instrumented code without satisfying this dependency will cause your program to fail on any instrumented code. To resolve this dependency add cobertura.jar to a location on your project's runtime classpath.
Web applications should place cobertura.jar in /WEB-INF/lib
I also strongly recommend placing a file named cobertura.properties somewhere at the root level of your applications runtime classpath with the following content:
Where ${basedir} is the location where you wish output from cobertura to be placed.
Technically, the net.sourceforge.cobertura.datafile property can be set in any manner, the properties file method tends to be the cleanest. Without this property output will be dumped to the executing processes current working directory. The current working directory for a managed java environment such as a webapp container (e.g. Tomcat) can be found by using the command line tool pwdx. Most webapp container installations are run from their base directory, but this will depend on your environment!
Once you have placed the cobertura.jar and cobertura.properties (optional) files in their correct locations bundle your artifacts with the instrumented class files instead of the normal ones.
Most people will find the best benefit by running a suite of automated tests against their code. The larger the project, the more tests should be automated (ideally), or you could find yourself doing a lot of (error prone!) leg work. However, for very small projects or quick demonstrations, manual testing is still certainly an option.
Once you have run all your tests against the instrumented code it is time to stop collecting statistics. Cobertura adds a shutdown hook to the running JVM, so stopping the runtime in any manner will cause cobertura to output it's statistics to cobertura.ser
In many environments it is not desireable to stop the JVM, in these cases a workaround is provided. Per the Cobertura FAQ:
You only need to hit a chunk of code that executes the following:
try {
String className = "net.sourceforge.cobertura.coveragedata.ProjectData";
String methodName = "saveGlobalProjectData";
Class saveClass = Class.forName(className);
java.lang.reflect.Method saveMethod = saveClass.getDeclaredMethod(methodName, new Class[0]);
saveMethod.invoke(null,new Object[0]);
} catch (Throwable t) {
}
The use of reflection here is deliberate. This is so you do not introduce cobertura.jar as a compile time dependency. I recommend using a second source folder in your project that gets built only for coverage/testing builds. So this code doesn't even have to appear in your production build!
Now for the fun part! Once we have gathered all the statistics contained in cobertura.ser, run cobertura-report on it. When running cobertura-report, you will get the most mileage when providing the list of source folders as parameters. This allows a line by line view of the statistics!
cobertura-report.sh --destination coverage src