JaCoCo + Mockito + Android tests: Zero coverage reported

Question

I know there are quite a few questions (and answers) for this topic, but I've tried everything I found in SO and other sites and I haven't found a way to make JaCoCo include coverage for Android tests that use Mockito.

My problem: I want to use JaCoCo to generate code coverage of both Unit Test and Instrumentation Test (androidTest). I'm using Mockito to mock some of the classes. I found a sample in GitHub to use JaCoCo and used it as a starting point.

https://github.com/rafaeltoledo/unified-code-coverage-android

When I run the custom jacocoTestReport task included in that example, the code coverage report is properly generated and code coverage is at 100%. The report includes both unit test and android test. However, that sample is not using Mockito (which I need), so I added the following to app/build.gradle

dependencies {
 ...
 androidTestCompile 'org.mockito:mockito-android:2.10.0'
}

I added a very simple Java class called Util at app/src/main/java/net/rafaeltoledo/coverage/Util.java

public class Util {
    public int anIntMethod() {
        return 0;
    }
}

And added the following simple test to the existing android test at app/src/androidTest/java/net/rafaeltoledo/coverage/MainActivityTest.java

@Test
public void utilMethod() {
    Util util = Mockito.mock(Util.class);
    Mockito.doReturn(10).when(util).anIntMethod();
    assertThat(util.anIntMethod(), is(10));
}

When I run the jacocoTestReport again, code coverage drops to 88% and the report in fact shows the Util class was not covered by my tests, even though I clearly have a test that exercises that class.

(I wanted to add screenshots of the reports but I don't have enough reputation, so here's a link to the coverage report and execution report that shows that both tests were in fact executed)

Versions info: Gradle plug-in: 2.3.3 Jacoco: 0.7.8.201612092310 Android Studio: 2.3.3 Android build tools: 25.0.2

Is this a Jacoco limitation or am I doing something wrong?

Answer 1

am I doing something wrong?

Let's put aside Android, because there is IMO clearly something wrong with your expectations/understanding about core thing here - mocking:

even though I clearly have a test that exercises that class.

By

@Test
public void utilMethod() {
    Util util = Mockito.mock(Util.class);
    Mockito.doReturn(10).when(util).anIntMethod();
    assertThat(util.anIntMethod(), is(10));
}

you are not testing anIntMethod, you are testing something that always returns 10, no matter what is actually written in anIntMethod.

Coverage shows what was executed and hence absolutely correct that it is zero for anIntMethod since it is not executed.

Mocking is used to isolate class under test from its dependencies, but not to replace it, otherwise you're not testing real code.

Here is an example of proper usage of mocking:

src/main/java/Util.java:

public class Util {
  int anIntMethod(Dependency d) {
    return d.get();
  }
}
class Dependency {
  int get() {
    return 0;
  }
}

src/test/java/UtilTest.java:

import org.junit.Test;
import org.mockito.Mockito;

import static org.junit.Assert.assertEquals;

public class UtilTest {
  @Test
  public void utilMethod() {
    Dependency d = Mockito.mock(Dependency.class);
    Mockito.doReturn(10).when(d).get();
    assertEquals(10, new Util().anIntMethod(d));
  }
}

build.gradle:

apply plugin: "java"
apply plugin: "jacoco"

repositories {
  mavenCentral()
}

dependencies {
  testCompile "junit:junit:4.12"
  testCompile "org.mockito:mockito-core:2.10.0"
}

And after execution of gradle build jacocoTestReport coverage is

And case of partial mocking:

src/main/java/Util.java:

public class Util {
  int get() {
    return 0;
  }

  int anIntMethod() {
    return get();
  }
}

src/test/java/UtilTest.java:

import org.junit.Test;
import org.mockito.Mockito;

import static org.junit.Assert.assertEquals;

public class UtilTest {
  @Test
  public void utilMethod() {
    Util util = Mockito.mock(
      Util.class,
      Mockito.withSettings().defaultAnswer(Mockito.CALLS_REAL_METHODS)
    );
    Mockito.doReturn(10).when(util).get();
    assertEquals(10, util.anIntMethod());
  }
}

In both cases mocked parts are shown as uncovered and this is correct.