Test failing when using JaCoCO

Question

I'm trying to measure coverage using JaCoCo on this project: https://github.com/square/retrofit

Everything seems to work fine but, for some reason, several tests that used to work, fail when using JaCoCo runtime agent.

Here is (the interesting part) of my pom.xml:

...
<plugins>
   ...
   <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.17</version>
      <executions>
         <execution>
            <id>default-test</id>
            <phase>test</phase>
            <goals>
               <goal>test</goal>
            </goals>
         </execution>
      </executions>
      <configuration>
         <!-- Sets the VM argument line used when unit tests are run. -->
         <argLine>${surefireArgLine}</argLine>
      </configuration>
   </plugin>
   ...
   <plugin>
      <groupId>org.jacoco</groupId>
      <artifactId>jacoco-maven-plugin</artifactId>
      <version>0.8.0</version>
      <executions>
         <!--
            Prepares the property pointing to the JaCoCo runtime agent which
            is passed as VM argument when Maven the Surefire plugin is executed.
        -->
         <execution>
            <id>pre-unit-test</id>
            <goals>
               <goal>prepare-agent</goal>
            </goals>
            <configuration>
               <!--
                    Sets the name of the property containing the settings
                    for JaCoCo runtime agent.
                -->
               <propertyName>surefireArgLine</propertyName>
            </configuration>
         </execution>
         <!--
            Ensures that the code coverage report for unit tests is created after
            unit tests have been run.
        -->
         <execution>
            <id>post-unit-test</id>
            <phase>test</phase>
            <goals>
               <goal>report</goal>
            </goals>
         </execution>
      </executions>
   </plugin>
   ...
</plugins>

The complete pom.xml can be found here: https://pastebin.com/HSKJpS3g

All the tests are failing for the same cause, let me show an example. Consider the class Example declared in this test:

@Test public void customMethodNoBody() {
    class Example {
      @HTTP(method = "CUSTOM1", path = "/foo")
      Call<ResponseBody> method() {
        return null;
      }
    }

    /* Do some operations with the class Example */
  }

Later in the code Exemple.class is passed to the method TestingUtils.onlyMethod which will throw an exception:

package retrofit2;

import java.lang.reflect.Method;

public final class TestingUtils {
  public static Method onlyMethod(Class c) {
    Method[] declaredMethods = c.getDeclaredMethods();
    if (declaredMethods.length == 1) {
      return declaredMethods[0];
    }
    throw new IllegalArgumentException("More than one method declared.");
  }
}

All the tests fail for this same exception being thrown, which wasn't happening before adding JaCoCo. Why is this happening ? How can I solve the problem ?

The command used by maven to run the tests is this:

/usr/lib/jvm/java-8-oracle/jre/bin/java
-javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.7.5.201505241946/org.jacoco.agent-0.7.5.201505241946-runtime.jar=destfile=/root/retrofit/retrofit/target/jacoco.exec
-jar /root/retrofit/retrofit/target/surefire/surefirebooter7714471086789859732.jar
/root/retrofit/retrofit/target/surefire/surefire2679778491836039056tmp
/root/retrofit/retrofit/target/surefire/surefire_02232907551688610770tmp

Note the -javaagent option added by JaCoCo. If you would like to have a look at the whole logs issued by mvn clean test, you can find them here: https://pastebin.com/kUtLtyjw

Answer 1

When using JaCoCo you want to ensure that syntethic fields and methods are not processed by your own code as JaCoCo uses them to collect the coverage statistics. Luckily both Field and Method have the isSynthetic() method.

public static Method onlyMethod(Class c) {
  return Arrays.stream(c.getDeclaredMethods())
    .filter(m -> !m.isSynthetic())
    .reduce((a, b) -> {
        throw new IllegalStateException("More than one method declared.");
    })
    .get();
}