tests under tox don't necessarily use the installed code

Question

The first feature listed on the tox website is "checking your package installs correctly with different Python versions and interpreters". This makes me think that if I screw up my setup.py, that the tests won't pass, and I'll be alerted to the fact that my setup.py has gone bad.

This isn't the case. What in fact happens is that the code is imported from the local directory (aka {toxinidir}) if you follow the usual pattern of putting your module or package in the same directory as your tox.ini. This means you can make your setup.py do absoluetely nothing, and tox will tell you its fine. At that point, it's quite easy to not notice the problem till after you've pushed to pypi and try to use the thing. This is the kind of issue that I'd like tox to prevent.

The chief issue is that the empty-string appears on sys.path during testing. Python interprets this to mean the present working directory and imports from there.

Is there any way to configure tox such that the local directory is not used during tests?

Currently my workaround is to cd {envtmpdir} && coverage run && mv .coverage {toxinidir}, but this is obviously an ugly hack.

Here I provide two versions of our code:

• A reliable reproduction of the issue.
• Our workaround.

My goal is to find a tox configuration that avoids the above problem, and is reasonable enough to recommend for use in all projects.

Answer 1

I have noticed that nosetests modifies sys.path during its run: It is adding the checked out code to it at the very beginning. This essentially breaks the separation tox is trying to create. To fix this, you just have to append the -P flag to nosetests:

[tox]
envlist = py27

[testenv]
commands=
    nosetests -P tests
deps=
    nose
    -rrequirements.txt

Also, I had to append tests (pointing nose to the tests/ directory) to prevent nose from importing any of the non-test code (I am using C-extensions that may or may not be built in the checkout).