Why doesn't my current directory show up in the path using pytest on Windows?

Question

I have the following folder structure;

myapp\
  myapp\
     __init__.py
  tests\
     test_myapp.py

and my pwd is

C:\Users\wwerner\programming\myapp\

I have the following test setup:

import sys
import pprint

def test_cool():
    pprint.pprint(sys.path)
    assert False

That produces the following paths:

['C:\\Users\\wwerner\\programming\\myapp\\tests',
 'C:\\Users\\wwerner\\programming\\envs\\myapp\\Scripts',
 'C:\\Windows\\system32\\python34.zip',
 'C:\\Python34\\DLLs',
 'C:\\Python34\\lib',
 'C:\\Python34',
 'C:\\Users\\wwerner\\programming\\envs\\myapp',
 'C:\\Users\\wwerner\\programming\\envs\\myapp\\lib\\site-packages']

And when I try to import myapp I get the following error:

ImportError: No module named 'myapp'

So it looks like it's not adding the current directory to my path.

By changing my import line to look like this:

import sys
sys.path.insert(0, '.')
import myapp

I am then able to import myapp with no problems.

Why does my current directory not show up in the path when running pytest? Is my only workaround to insert . into the sys.path? (I'm using Python 3.4 if it matters)

Answer 1

Ahah!

After comparing the layout of my cookiecutter repo, it turns out to be way more simple (and better) than that.

tests/
    __init__.py
    test_myapp.py

A simple addition of the __init__.py file to my test dir allows me to run py.test from my main directory.

Answer 2

Using an installable package

If you have an installable package (setup.py or pyproject.toml file with a build-system defined) then you want to test against the installed package.

pip install --editable .
pytest

The simplest possible way to make the project shown in the question into an installable package would be by adding this setup.py:

from setuptools import setup

setup(
    name="myapp",
    version="0.1",
    packages=["myapp"],
)

This will put the myapp code at /path/to/myapp/.venv/lib/python3.XY/site-packages, which is in the sys.path of the virtual environment. Now myapp can be imported from the site, and it is neither necessary nor desirable for the current working directory to be present on sys.path during test execution.

Not using an installable package

The project shown in the question does not have any installer, so it can't be installed. It can still be tested by making sure the project root (i.e. the directory which contains both myapp and tests as subdirectories) is present on sys.path.

The best way to do this is not to invoke the bare pytest command, but use python -m pytest instead. That adds the current working directory to the start of sys.path. It's the normal Python behavior when executing a package as __main__ (documented here) and it's also a documented usage for pytest - see Invoking pytest versus python -m pytest.

Why does adding an __init__.py to the tests subdirectory (not) work?

The directory structure shown in the question is the "Tests outside application code" pattern, documented here. This is also the directory structure I recommend, since it creates a clear distinction between library/application code and test code.

It's not recommended to add __init__.py files inside the test directories when using a "Tests outside application code" structure, since the test files aren't intended to be "packaged" (e.g. test files do not really need to import from other test files, and they do not need to be installed at all for end users of your package).

The reason adding a myapp/__init__.py actually allows myapp to be imported by pytest, as described in Wayne's answer is actually an accident due to the way test discovery appends sys.path during the test collection phase. This is described as "problematic" in the docs

... this introduces a subtle problem: in order to load the test modules from the tests directory, pytest prepends the root of the repository to sys.path, which adds the side-effect that now mypkg is also importable

They go on to strongly recommend using the src-layout if you intend to have __init__.py files inside test directories, to avoid this confusion of the import system.

But perhaps the best reason not to rely on this side-effect is that pytest collection actually can work in multiple modes (see import modes), and Wayne's answer relies upon pytest using the default "prepend" mode. It is currently mentioned that a future version will switch to "importlib" mode as default:

We intend to make importlib the default in future releases.

The accepted answer does not work with pytest --import-mode=importlib and so will stop working altogether at some stage.