Why do Intellij code coverage and jacoco code coverage show different percentages?

Question

I have a gradle project(java) in intellij. I right clicked the project in intellij and ran Run Tests in projectName with coverage which created some test report on right hand side. In that right hand side I have numbers like

| Class, %   | Method, %   | Line, %
--------------------------------------
80%(80/100)  50%(100/200)  30%(30/100)

Note: The above numbers are just for example. These are not real.

Now I went to command line and ran gradlew jacocoTestReport which gave a different set of numbers for Method and Line, but Class numbers were same. Why is there a discrepancy in this case?

Is there a way to run intellij's code coverage from command line instead of right clicking?

I just want to know if Intellij uses a different way to calculate these numbers as opposed to jacoco. But even in that case my assumption is that there is only one way to calculate stuff right? Or does intellij or jacoco doesn't count classes that has Lombok annotations etc thereby reducing the number of methods(getters and setters) in the final count?

Answer 1

I stumbled over this question on a related mission and although it's old, I figure it may still be interesting to someone.

Method numbers

The main question is if the intellij coverage and jacoco calculate the numbers differently, and which way is correct. Brief answer: The intellij coverage summary uses the methods directly supplied by the developer, while jacoco operates on bytecode level and displays the number of methods found there. To demonstrate this, I created a simple class with four methods:

public class Exp {

    private final LinkedList<Integer> vals = new LinkedList<>();

    public void addVal(int v) {
        vals.add(v);
    }

    public List<Integer> doubled() {
        return vals.stream().map(x -> x*2).collect(Collectors.toList());
    }

    public List<Integer> evens() {
        return vals.stream().filter(x -> x%2 == 0).collect(Collectors.toList());
    }

    public static void main(String[] args) {
        Exp t = new Exp();

        t.addVal(1);
        t.addVal(2);
        t.addVal(3);

        System.out.println(t.doubled());
        System.out.println(t.evens());
    }
}

In the intellij summary displays the following values on the right:

So the number of methods equals the number of methods in the example code. Jacoco reports seven methods, as can be seen in the report (same as in the Emma plugin in eclipse 2020-09):

This is the number of methods we can find in the bytecode, e.g. by using the javap disassembler command. Here we see that the two lambda expressions are implemented as methods of the class, and also a standard constructor is inserted.

C:\_workspace\intellij\Tests\out\production\mod>javap -p Exp.class
Compiled from "Exp.java"
public class Exp {
  private final java.util.LinkedList<java.lang.Integer> vals;
  public Exp();
  public void addVal(int);
  public java.util.List<java.lang.Integer> doubled();
  public java.util.List<java.lang.Integer> evens();
  public static void main(java.lang.String[]);
  private static boolean lambda$evens$1(java.lang.Integer);
  private static java.lang.Integer lambda$doubled$0(java.lang.Integer);
}

What puzzles me a little is that the intellij coverage report (Run->Generate Corevage Report) displays five methods:

Adding a standard constructor to the code and re-generating the report reveals that the report included the generated standard constructor, but not the lambda expressions. There seems to be an intermediate counting method.

As for the question if intellij or jacoco is right I would say that they both are, it's just a question of definition.

Line numbers

In my tests all reports displayed consistent line numbers. In the example above 13 lines containing executable code were reported. My impression of the intellij line count in the coverage summary is that it does not refresh properly all the time. A clean rebuild may have been necessary.