Using pytest_addoptions in a non-root conftest.py

Question

I have a project that has the following structure:

Project/
|
+-- src/
|   |
|   +-- proj/
|       |
|       +-- __init__.py
|       +-- code.py
|       +-- tests/
|           |
|           +-- __init__.py
|           +-- conftest.py
|           +-- test_code.py
+-- doc/
+-- pytest.ini
+-- setup.py

The importable package proj lives under the src directory. Tests are installed along with it, in the proj.tests sub-package. The code, the tests themselves, and the installation all work fine. However, I have trouble passing options to the tests.

I have an option, --plots, that is defined in conftest.py (under Project/src/proj/tests/), which creates a .plots/ folder in the root folder and places some graphical debugging there:

# conftest.py
def pytest_addoption(parser):
    parser.addoption("--plots", action="store_true", default=False)

There are two different ways I'd like to have to run this test:

1. The first is from the command line, in the Project directory:

    $ pytest --plots
   

   This fails immediately with

    usage: pytest [options] [file_or_dir] [file_or_dir] [...]
    pytest: error: unrecognized arguments: --plots
      inifile: /.../Project/pytest.ini
      rootdir: /.../Project
   

   If I add the package directory, the run goes fine:

    $ pytest src/proj --plots
    ...
    rootdir: /.../Project, inifile: pytest.ini
    ...
   

   It also goes well if I specify the tests sub-directory:

    $ pytest src/proj/tests --plots
    ...
    rootdir: /.../Project, inifile: pytest.ini
    ...
   

2. The second way is to run the test function of the package itself. The test function is defined in Project/src/proj/__init__.py like this:

    # __init__.py
   
    def test(*args, **kwargs):
        from pytest import main
        cmd = ['--pyargs', 'proj.tests']
        cmd.extend(args)
        return main(cmd, **kwargs)
   

   The function is intended to be run after Project is installed like this:

    >>> import proj
    >>> proj.test()
   

   Generally, this works OK, but if I do

    >>> proj.test('--plots')
   

   I get the same error as usual:

    usage:  [options] [file_or_dir] [file_or_dir] [...]
    : error: unrecognized arguments: --plots
      inifile: None
      rootdir: /some/other/folder
   

   For this test, I ran python setup.py develop and then cd'dto/some/other/folder` to make sure everything installed correctly.

My goal is to have both options working correctly when I pass in the --plots command line option. It seems that I have found a workaround for option #1, which is to manually pass in one of the packages to pytest, but I don't even fully understand how that works (why can I pass in either src/proj or src/proj/tests?).

So the question is, how to I get pytest to consistently run my test package so that the correct conftest.py gets picked up? I am willing to consider just about any reasonable alternative that allows me to get the --plots option working consistently in both the source (running in a shell from Project) version and the proj.test() version.

---

For reference, here is my pytest.ini file:

# pytest.ini

[pytest]
testpaths = src/proj/tests
confcutdir = src/proj/tests

It doesn't appear to make any difference.

I am running with pytest 3.8.0, on Pythons 2.7, 3.3, 3.4, 3.5, 3.6 and 3.7 (in anaconda). In all versions, the results are reproducible.

Answer 1

Quote from the pytest_addoption docs:

Note: This function should be implemented only in plugins or conftest.py files situated at the tests root directory due to how pytest discovers plugins during startup.

...

Note that pytest does not find conftest.py files in deeper nested sub directories at tool startup. It is usually a good idea to keep your conftest.py file in the top level test or project root directory.

When you call pytest from the Project dir, pytest recognizes it as the root dir, so the addoption hook in Project/src/proj/tests/conftest.py is ignored.

If the project structure you described is a requirement, I'd move the pytest_addopts hook impl to a separate plugin module; this can reside anywhere in the project dir, for example Project/src/proj/tests/plugin_plots.py. Now, if you call

$ pytest -p src.proj.tests.plugin_plots

the addoption hook impl will be executed and the args will be added correctly.

(you may need to adjust the sys.path to be able to load the custom plugin module correctly, e.g. use PYTHONPATH=. pytest -p or python -m pytest -p, or install the project in editable mode etc)

To not to enter the -p arg each time, persist it in pytest.ini, e.g.

# pytest.ini
[pytest]
addopts=-p src.proj.tests.plugin_plots

Edit:

When running the tests programmatically, ehnance the args list:

def test(*args, **kwargs):
    from pytest import main
    cmd = ['-p', 'proj.tests.plugin_plots', '--pyargs', 'proj.tests']
    cmd.extend(args)
    return main(cmd, **kwargs)

should work just fine.