Integration Test Coverage in SonarQube from JaCoCo Maven plug-in showing 0%

Question

We have a multi-module multi-language maven java project with coverage analysis with jacoco. The main part of the modules is backend (Java code) with a REST API and our webapp module contains the frontend (AngularJS) and the Integration-Tests in java. Our Jacoco-IT.exec file contains around 100Kb of data so we guess some Coverage data for the Integration tests could get collected. Nevertheless we cant see any coverage in SonarQube (using Version 5.0 and 4.5)

We run build the Project and run the integration tests with

mvn clean install

and analyse the data with

mvn sonar:sonar

The Project is configured as multi language, but we just analyse the java code yet (we used to have a monolanguage configuration but there was no difference in the results related to the coverage data)

Our parent POM:

<properties>
    <hibernate.version>4.2.2.Final</hibernate.version>
    <spring.version>3.2.3.RELEASE</spring.version>
    <resteasy.version>3.0.4.Final</resteasy.version>
    <cucumber.version>1.1.3</cucumber.version>
    <selenium-java.version>2.33.0</selenium-java.version>
    <!-- Sonar -->
    <sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
    <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
    <sonar.jacoco.itReportPath>${project.basedir}/../target/jacoco-it.exec</sonar.jacoco.itReportPath>
    <jacoco.dataFile>${project.basedir}/../target/jacoco-it.exec</jacoco.dataFile>
</properties>
<build>
   <plugins>
       <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.7</source>
                <target>1.7</target>
                <debug>true</debug>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>

            <executions>
                <execution>
                    <id>default-prepare-agent</id>
                    <goals>
                        <goal>prepare-agent</goal>
                    </goals>
                </execution>
                <execution>
                    <id>default-prepare-agent-integration</id>
                    <goals>
                        <goal>prepare-agent-integration</goal>
                    </goals>
                </execution>
                <execution>
                    <id>default-report</id>
                    <goals>
                        <goal>report</goal>
                    </goals>
                </execution>
                <execution>
                    <id>default-report-integration</id>
                    <goals>
                        <goal>report-integration</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.16</version>
        </plugin>
    </plugins>
    <pluginManagement>
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>0.7.2.201409121644</version>
            </plugin>
    </pluginMangement>
</build>

Our POM from the webapp with the Integration Tests:

<properties>
    <sonar.sources>src/main</sonar.sources>
</properties>
<build>
    <finalName>web</finalName>
    <plugins>
        <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>8.1.7.v20120910</version>
        <configuration>...</configuration>
        <executions>...</executions>
    </plugin>
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <configuration>
                <!-- The destination file for the code coverage report has to be set to the same value
                in the parent pom and in each module pom. Then JaCoCo will add up information in
                the same report, so that, it will give the cross-module code coverage. -->
                <destFile>${project.basedir}/../target/jacoco-it.exec</destFile>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.16</version>
            <executions>
                <execution>
                    <id>default</id>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

As you can see in our parent POM we configured the itReportPath to be in the same directory for all Modules. So when Sonar analyses all modules it reads always the same coverage data and should align them with the debug information from the binaries of the current module. During the analyse phase of maven (mvn sonar:sonar) we get some messages where we are not sure if they are a problem:

for all java modules we get the folloing lines:

    [INFO] [17:30:42.992] Sensor SurefireSensor...
    [INFO] [17:30:42.992] parsing \git\airport-core\rest\target\surefire-reports
    [INFO] [17:30:43.009] Sensor SurefireSensor done: 17 ms
    [INFO] [17:30:43.009] Sensor JaCoCoItSensor...
    [INFO] [17:30:43.010] Analysing \git\airport-core\rest\..\target\jacoco-it.exec
    [INFO] [17:30:43.018] No information about coverage per test.
    [INFO] [17:30:43.018] Sensor JaCoCoItSensor done: 9 ms
    [INFO] [17:30:43.018] Sensor JaCoCoOverallSensor...
    [INFO] [17:30:43.027] Analysing \git\airport-core\rest\target\sonar\jacoco-overall.exec
    [INFO] [17:30:43.046] No information about coverage per test.
    [INFO] [17:30:43.046] Sensor JaCoCoOverallSensor done: 28 ms

Is "No information about coverage per test." a Problem? Ok, we dont know which test caused the coverage, but we should get a different value in SonarQube than 0%

and for our webapp module, which does not contain any java sources/classes except for the integration test sources itself, we get the following lines:

    [INFO] [17:30:48.370] Sensor JaCoCoItSensor...
    [INFO] [17:30:48.370] No JaCoCo analysis of project coverage can be done since there is no class files.
    [INFO] [17:30:48.370] Sensor JaCoCoItSensor done: 0 ms

This shouldn't be a problem, as there are no classes in this module. Is this observation correct?

To the Main part our configruation bases on the official SonarCube demo configuration (https://github.com/SonarSource/sonar-examples/tree/master/projects/languages/java/code-coverage/combined%20ut-it/combined-ut-it-multimodule-maven-jacoco), we just added the "prepare-agent-integration" goal. And of course changed it to a multi-language Project.

We found a lot of depricated information on the web (argLine settings and so on) which didn't change anything for us.

Links to the Documentation and Tutorials we used:

• http://docs.sonarqube.org/display/SONAR/Code+Coverage+by+Integration+Tests+for+Java+Project
• http://www.sonarqube.org/measure-coverage-by-integration-tests-with-sonar-updated/

Answer 1

I had the same experience when I was configuring JaCoCo for Jersey. I ended up applying the bisection method in order to find out which modules are causing the code coverage for the whole project to drop to 0%.

If I remember it correctly, the biggest surprise was maven-shade-plugin which somehow corrupted the association between the collected coverage and the location of classes which caused Sonar to show 0% code coverage. I ended up disabling its execution by binding it to phase none in order to fix it (see this commit).

As an example, this is how to extend a pom.xml of a sub-module in order to have working sonar (assuming, sonar is executed with sonar maven profile).

<profile>
    <id>sonar</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <!-- disable parallel execution so that JaCoCo listener can properly work -->
                    <parallel>none</parallel>
                    <perCoreThreadCount>false</perCoreThreadCount>
                    <forkCount>1</forkCount>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-shade-plugin</artifactId>
                <executions>
                    <execution>
                        <id>shade-archive</id>
                        <phase>none</phase>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>org.ow2.asm</groupId>
            <artifactId>asm-debug-all</artifactId>
            <optional>false</optional>
        </dependency>
    </dependencies>
</profile>

Answer 2

Im using a similar setup and everything works perfectly with the following configuration:

However the maven-surefire-plugin was causing troubles initially. Thats why I had to add the @{argLine} in the argLine tag, since the jacoco-maven-plugin:prepare-agent goal is executed before the maven-surefire-plugin. (see https://stackoverflow.com/a/25774988)

Hope I could help

<plugins>
    <plugin>
        <groupId>org.sonarsource.scanner.maven</groupId>
        <artifactId>sonar-maven-plugin</artifactId>
        <version>3.4.0.905</version>
    </plugin>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>

        <executions>
            <execution>
                <phase>test</phase>
            </execution>
        </executions>
        <configuration>
            <!-- the @{argLine} is needed for proper jacoco execution -->
            <argLine>@{argLine} -Xmx256m</argLine>
        </configuration>
    </plugin>

    <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>0.7.9</version>
        <executions>
            <execution>
                <id>jacoco-initialize</id>
                <goals>
                    <goal>prepare-agent</goal>
                </goals>
            </execution>
            <!-- generate fancy html report -->
            <execution>
                <id>jacoco-site</id>
                <phase>package</phase>
                <goals>
                    <goal>report</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
</plugins>