How can I enable parallelism via pytest -nauto if pytest-xdist is installed?

Question

To enable parallel tests, one has to install pytest-xdist and use pass the -nauto option to pytest to use all available CPUs. I would like to enable -nauto by default, but still make pytest-xdist optional. So this would not work:

[pytest]
addopts = -nauto

Is there a way to enable pytest parallelism by default if pytest-xdist is installed? (It should also be possible to disable it again with pytest -n0 if desired.)

I guess that some kind of conftest.py hook has to be written? It is possible to detect installed plugins, but pytest_configure is ran after loading plugins which might be too late. Besides, I am not sure how options can be added at that point (or how to configure directly manipulate xdist).

Answer 1

You can check whether the xdist options group has numprocesses arg defined. This indicates that pytest-xdist is installed and the option will be handled. If this is not the case, your own dummy arg will ensure the option is known to pytest (and safely ignored):

# conftest.py

def pytest_addoption(parser):
    argdests = {arg.dest for arg in parser.getgroup('xdist').options}
    if 'numprocesses' not in argdests:
        parser.getgroup('xdist').addoption(
            '--numprocesses', dest='numprocesses', metavar='numprocesses', action='store',
            help="placeholder for xdist's numprocesses arg; passed value is ignored if xdist is not installed"
    )

Now you can leave the option in the pytest.ini even if pytest-xdist is not installed; however, you will need to use the long option:

[pytest]
addopts=--numprocesses=auto

The reason for that is that short options are reserved for pytest itself, so the code above doesn't define or use it. Should you really need the short option, you have to resort to the private method:

parser.getgroup('xdist')._addoption('-n', '--numprocesses', dest='numprocesses', ...)

Now you can use the short option in the config:

[pytest]
addopts=-nauto