LCOV/GCOV branch coverage with C++ producing branches all over the place

Question

We are using LCOV/GCOV to produce test coverage of our projects. Recently we tried to enable branch-coverage additionally. But it looks like, this just doesn't yield the results we expected from a high-level developer view.

Using branch-coverage with C++ blows the report up with branches all over the place. We suspect (as the searching for the issues indicates) that mostly exception handling code creates these "hidden branches". And GCOV/LCOV doesn't seem to skip over these.

I created a small test project to show the problem: https://github.com/ghandmann/lcov-branch-coverage-weirdness

Currently we use Ubuntu 16.04. with:

• gcc v5.4
• lcov & genhtml v1.12

Our production code is built with c++11 enabled. The minimal example isn't built with c++11 enabled, but as we experimented a bit with all different options (c++ standard, optimization, -fno-exceptions) we didn't come up with a passable result.

Anybody got some ideas? Tipps? Are we using anything the wrong way? Is this - as stated somewhere else - really expected behavior?

Update:

As also pointed out on the gcc-help mailing list, these "hidden branches" occur because of exception handling. So adding the -fno-exceptions switch to gcc produces 100% branch coverage for "simple" programs. But when exceptions are disabled, gcc refuses to compile code which actually uses exceptions (e.g. try-catch, throw). Therefore for real production code this is not an option. Looks like, you have to simply declare ~50% coverage to be the new 100% in this case. ;)

Answer 1

The thing is that GCC also records branch information for each line where a scope exit due to some thrown exception is possible (e.g. on Fedora 25 with GCC 6.3.1 and lcov 1.12).

The value of this information is limited. The main use case for branch coverage data are complicated if-statements that have a multi-clausal logical expression like this:

if (foo < 1 && (bar > x || y == 0))

Say you are interested to verify whether your test suite also covers the bar > x case or if you just have test cases where y == 0.

For this, branch coverage data collection and the visualization by lcov's genhtml is useful. For simple if-statements like

if (p == nullptr) {
  return false;
}
return true;

you don't need branch coverage data because you see whether the branch was taken or not via looking at the coverage of the following lines.

The input of genhtml that is generated by lcov is in a relatively simple text format (cf. geninfo(1)). Thus, you can post-process it such that all lines that start with BRDA: and don't belong to an if-statement are removed. See for example filterbr.py which implements this approach. See also gen-coverage.py for the other lcov/genhtml processing steps and an example project where the resulting trace file is uploaded to codecov (codecov doesn't use genhtml but can import lcov trace files and displays branch coverage data).

(Non-)Alternatives

• disabling exceptions is only an option when your C++ code doesn't use any
• compiling with something like -O1 -fno-omit-frame-pointer -fno-optimize-sibling-calls somewhat reduces the number of recorded branch coverage data, but not much
• Clang supports GCOV style coverage collection but also implements a different approach, called 'source-based code coverage' (compile with -fprofile-instr-generate -fcoverage-mapping and post-process with llvm-profdata and llvm-cov). That toolchain doesn't support branch coverage data, though (as of 2017-05-01).
• by default, lcov+genhtml don't generate branch coverage data - sometimes you don't really need it (see above), thus, it is then an option to disable it (cf. --rc lcov_branch_coverage=0 and --no-branch-coverage)

Answer 2

GCC will add a bunch of exception handling stuff. Especially when you do function calls.

You can fix this by adding -fno-exceptions -fno-inline to your build.

I should add, you probably only want these flags on for testing. So something like this:

g++ -O0 --coverage -fno-exceptions -fno-inline main.cpp -o test-coverage 

Answer 3

There's a PR in progress that addresses these limitations. https://github.com/linux-test-project/lcov/pull/86.

This paper explains the theory behind the implementation.