I'm working on a CLI program using click, and I want to start adding some tests with code coverage analysis using coverage.py.
I thought a good way to implement the tests would be to run the CLI itself using subprocess. However, coverage.py reports zero code coverage, presumably because the Python instance spawned by subprocess doesn't have the coverage.py instrumentation.
I found this link that says I can drop a sitecustomize.py
file in my PYTHONPATH to always force Python to start coverage measurement, but I'm using Tox to create a venv and run tests. I couldn't find any Tox settings that deal with this.
I found this answer that says I should just run my CLI through coverage run
, but it looks like that only works if given a path to a Python script, and I'm trying to run my CLI through the entry point defined in setup.py
. i.e. I have to change all my command lines in test code from myprogram
to coverage run myprogram/cli/cli.py
. I'd rather not do this because it's not the way I expect users to run the program.
So it seems like the two options are:
figure out a way to make sitecustomize.py
work in Tox environments, or
all command lines in test code to use a script path instead of an entry point (probably easier in the long run, but makes tests slightly more brittle and harder to understand). Wondering if there's anything else I'm missing.
CliRunner and a proper unit test framework is the way to go. Here is an example setup for PyTest which uses the PyTest Coverage plugin
from click.testing import CliRunner
from click_prog import hello
def test_hello_world():
runner = CliRunner()
result = runner.invoke(hello, ['--opt', 'An Option', 'An Arg'])
assert result.exit_code == 0
assert result.output == 'Opt: An Option Arg: An Arg\n'
result = runner.invoke(hello, ['An Arg'])
assert result.exit_code == 0
assert result.output == 'Opt: None Arg: An Arg\n'
if __name__ == '__main__':
test_hello_world()
import click
import sys
@click.command()
@click.option('--opt')
@click.argument('arg')
def hello(arg, opt):
"""A Simple program"""
click.echo('Opt: {} Arg: {}'.format(opt, arg))
if __name__ == '__main__':
hello(sys.argv[1:])
[pytest]
# -- recommended but optional:
# python_files = tests.py test_*.py *_tests.py
===================== test session starts =====================
platform darwin -- Python 3.6.5, pytest-3.7.1, py-1.5.4, pluggy-0.7.1
rootdir: /Users/strauch/dev/fix_windows, inifile: pytest.ini
plugins: xdist-1.22.5, forked-0.2, cov-2.6.0
collected 1 item
test_click.py . [100%]
---------- coverage: platform darwin, python 3.6.5-final-0 -----------
Name Stmts Miss Cover
------------------------------------
click_prog.py 8 1 88%
test_click.py 12 1 92%
------------------------------------
TOTAL 20 2 90%
================== 1 passed in 0.07 seconds ===================
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With