clover: how does it work?

Question

I am evaluating clover currently and wonder how to use it best. First I'd like to understand how it works conceptually.

1) What does instrumentation mean? Are the test-calls attached to implementation's statements?

2) How is this done? Are the tests actually executed with some fancy execution context (similar to JRebel e.g.) for this? Or is it more like static analysis ?

3) After a "clover-run", some DB is saved to disk, and based on this, reports are generated right? Is the DB-Format accessible? I mean Can I launch my own analysis on it, e.g. using my own reporting tools ? What information does the DB contain exactly? Can I see the mapping between test and implementation there ?

4) Are there other tools that find the mapping between test and implementation? Not just the numbers, but which test, actually covers a line of code ...

Thanks, Bastl.

Answer 1

How is this done? Are the tests actually executed with some fancy execution context (similar to JRebel e.g.) for this? Or is it more like static analysis?

During code instrumentation by Clover it detects which methods are test methods (by default it recognizes JUnit3/4 and TestNG). Such methods gets additional instrumentation instructions. Shortly speaking, entering a test method will usually instantiate a dedicated coverage recorder which measures coverage exclusively for this test. More information about per test recording strategies available in Clover:

• https://confluence.atlassian.com/display/CLOVER/Clover+Performance+Results
• https://confluence.atlassian.com/display/CLOVER/About+Distributed+Per-Test+Coverage

After a "clover-run", some DB is saved to disk, and based on this, reports are generated right?

A Clover database (clover.db) contains information about code structure (packages, files, classes, methods, statements, branches), it has also information about test methods. There are also separate coverage recording files (produced at runtime) containing information about number of "hits" of given code element. Clover supports both global coverage (i.e. for the whole run) as well as per-test coverage (i.e. coverage from a single test).

More information is here:

• https://confluence.atlassian.com/display/CLOVER/Managing+the+Coverage+Database

Is the DB-Format accessible?

The API is still in development (https://jira.atlassian.com/browse/CLOV-1375), but there is a possibility to get basic information. See:

• https://confluence.atlassian.com/display/CLOVER/Database+Structure

for more details about DB model and code samples.

But the question is: do you really need to manually read this DB? You wrote that:

Can I see the mapping between test and implementation there ?

Such mapping is already provided by Clover - in the HTML report for example if you click on a source line it will pop up a list of test methods hitting this line.

PS: I'm a Clover developer at Atlassian, feel free to contact me if you have any questions.

Answer 2

What does instrumentation mean?

Additional code is woven in with your code.

Are the test-calls attached to implementation's statements?

I am not sure what you mean but it could be instructions or call to methods. Trivial methods will be inlined by the JIT at runtime.

How is this done?

There are many ways to do it, but often the Instrumentation class is to used to capture when a class is being loaded and a library like Objectweb's ASM is used to manipulate the code.

Are the tests actually executed with some fancy execution context

The context counts which lines have been executed.

Or is it more like static analysis ?

No, it is based on what is called.

After a "clover-run", some DB is saved to disk, and based on this, reports are generated right? Is the DB-Format accessible?

You had best ask the producers of clover as to the content of their files.

Are there other tools that find the mapping between test and implementation? Not just the numbers, but which test, actually covers a line of code ...

There are many code coverage tools available including EMMA, JaCoCo, Cobertura, IDEA has one builtin.