Can python coverage module conditionally ignore lines in a unit test?

Question

Using nosetests and the coverage module, I would like coverage reports for code to reflect the version being tested. Consider this code:

import sys
if sys.version_info < (3,3):
    print('older version of python')

When I test in python version 3.5, the print() shows up as untested. I'd like to have coverage ignore that line, but only when I'm testing using python version 3.3+

Is there a way to do something like # pragma: no cover on the print() statement only for when sys.version_info is not less than (3,3)? Effectively, I'd like to do something like this:

import sys
if sys.version_info < (3,3):
    print('older version of python') # pragma: [py26,py27,py32] no cover

Answer 1

Another option is to use a different .coveragerc file for different versions of Python, and to set the exclude_lines regex differently for the different versions.

I've seen some people use a different comment string, # no cover 3.x vs # no cover 2.x, for example.

But keep in mind, you don't have to use a comment pragma at all. The regex is applied to the entire line. For example, if you use a short notation for your conditional, like:

if PY2:
    blah_py2_stuff_blah()

then your .coveragerc file for Python 3 could have:

[report]
exclude_lines =
    # pragma: no cover
    if PY2:

Then the if PY2: lines would be excluded without any extra comments or effort on your part.

Answer 2

As you explain in the comments, your concern is, that the coverage report will only show line numbers, and you want to avoid having to re-check these again and again.

On the other hand, I am not much in favor of cluttering code with comments to make one or the other tool happy: To me all this is degrading readability. Thus, I'd like to propose another approach, that avoids cluttering the code, but still take away the burden from you to do that re-checking all the time.

The idea is, to create a baseline of the coverage situation, against which you can compare future coverage analysis results. For example, the coverage report from coverage.py looks as follows (cited from http://coverage.readthedocs.org/en/coverage-4.0.3/index.html):

Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
my_program.py                20      4    80%   33-35, 39
my_other_module.py           56      6    89%   17-23
-------------------------------------------------------
TOTAL                        76     10    87%

This output could be used as the basis for a 'baseline': The rough idea (for improvements see below) is, that you store this output as the 'accepted' coverage situation, and diff it against future coverage reports. Unfortunately, whenever line numbers change, you will see differences when diffing the reports. To avoid this, this basic idea can be improved:

With the help of simple scripting, you could transform the report such that instead of the line numbers, the contents of the respectice lines are shown. For example, a hypothetical report based on your code example above could look like:

Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
my_program.py                20      1     5%   3

From this report, you could create the following 'coverage baseline' for python versions >= 3.3, for example in file coverage-baseline-33andabove.txt:

my_program.py:
-    print('older version of python')

This baseline would look the same even if you add, for example, further import lines to the top of your file. Further baseline files would be created for the other python versions against which you determine the coverage.

Further possible improvements could be to separate groups of lines, like:

my_program.py:
*
-    print('older version of python')
*
-    cleanup()
-    assert False
my_program2.py:
*
-    print('older version of python')

You would only see differences whenever the non-covered code changes (additions, deletions, modifications, moves) and also when the file names change. Then, the occurrence of differences will require you to store a new 'coverage-baseline', or alternatively, add more tests until the original baseline content is reached again.