I am adding unit tests and to a kind of "legacy" Python package. Some of the modules contain their own doctests embedded in docstrings. My goal is to run both those doctests and new, dedicated unit tests.
Following this Q&A ("How to make py.test run doctests as well as normal tests directory?") I'm using the --doctest-modules
option to pytest
. When running from the source repository, pytest
indeed discovers the embedded doctests from Python modules under the src
directory.
However, my goal is to test that the source distribution builds and installs at all, and then test everything against the installed package. To do this I'm using tox
which automate the process of building a sdist
(source distribution) tarball, installing it in a virtual environment, and running the tests against the installed version. To ensure that it is the installed version, rather than the one in the source repository, that is imported by the tests, I follow the suggestion in this article and the repository looks like this now:
repo/
src/
my_package/
__init__.py
module_a.py
module_b.py
...
tests/
test_this.py
test_that.py
requirements.txt
setup.py
tox.ini
(The test scripts under tests
import the package as in import my_package
, which hits the installed version, because the repository layout makes sure that the src/my_package
directory is out of the module search paths.)
And in the tox
configuration file, the relevant sections looks like
[tox]
envlist = py27,py36,coverage-report
[testenv]
deps =
-rrequirements.txt
commands =
coverage run -p -m pytest --
[pytest]
addopts = --doctest-modules
So far, the tests run fine, and the doctests are picked up -- from the modules under src/my_package
, rather than from the package installed in tox
virtual environments.
My questions related to this set-up is as follows:
tox
seems to ensure that what you install is what you have in the source repository, but does it?pytest
to actually run doctests from the installed modules in a sort of clean way? I can think of a few solutions such as building a dedicated documentation tree in a doc
directory and let pytest
find the doctests in there with the --doctest-glob
option. But is there a method to do this without building the docs first?pytest also introduces new options: ALLOW_UNICODE : when enabled, the u prefix is stripped from unicode strings in expected doctest output. This allows doctests to run in Python 2 and Python 3 unchanged.
Right click on a blank space in the python code, and there is a menu option to run all the Doctests found in the file, not just the tests for one function.
I found the answer to my own question.
To quote pytest
issue #2042:
currently
doctest-modules
is fundamentally incompatible with testing against a installed package due to module search in checkout vs module usage fromsite-packages
So as of now the solution does not exist.
pytest
collects tests from the current directory (unless you instruct it otherwise passing an explicit directory). Add the installation directory using tox substitutions in tox.ini
. I.e., either pass the directory:
[testenv]
deps =
-rrequirements.txt
commands =
coverage run -p -m pytest {envsitepackagesdir}/my_package
or change directory:
[testenv]
changedir = {envsitepackagesdir}/my_package
deps =
-rrequirements.txt
commands =
coverage run -p -m pytest --
This is how I solved the import mismatch:
$ pytest tests/ --doctest-modules --pyargs myrootpkg <other args>
The problem here is that once you start specifying source paths explicitly (in this case via --pyargs
), you have to specify all other source paths as well (tests/
in this example) as pytest
will stop scanning the rootdir. This shouldn't be an issue when using the src
layout though, as the tests aren't usually scattered around the repository.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With