Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pytest select tests based on mark.parameterize value?

Tags:

python

pytest

I see from here that I can pick out tests based on their mark like so:

pytest -v -m webtest

Let's say I have a test decorated like so:

@pytest.mark.parametrize('platform,configuration', (
    pytest.param('win', 'release')
    pytest.param('win', 'debug')))
def test_Foo(self):

I'd like to do something like the following:

pytest -v -m parameterize.configuration.release

That way I run test_Foo with the parameter of 'release' but not the parameter of 'debug'. Is there a way to do this? I think I can do this by writing a wrapper test and then passing only the desired parameter down, but I want to avoid doing so because we already have a large number of tests parameterized as describe, and I want to avoid writing a large number of wrapper tests.

like image 643
JuniorIncanter Avatar asked Oct 12 '18 19:10

JuniorIncanter


2 Answers

You can use -k for expression-based filtering:

$ pytest -k win-release

will run only tests containing win-release in their names. You can list all names without executing the tests by issuing

$ pytest --collect-only -q

Should an expression be not enough, you can always extend pytest by adding your custom filtering logic, for example by passing parameter name and value via command line args and selecting only tests that are parametrized accordingly:

# conftest.py

def pytest_addoption(parser):
    parser.addoption('--param-name', action='store',  help='parameter name')
    parser.addoption('--param-value', action='store',  help='parameter value')


def pytest_collection_modifyitems(session, config, items):
    param_name = config.getoption('--param-name')
    param_value = config.getoption('--param-value')
    if param_name and param_value:
        items[:] = [item for item in items
                    if hasattr(item, 'callspec')
                    and param_name in item.callspec.params
                    and item.callspec.params[param_name] == param_value]

Now you can e.g. call

$ pytest --param-name=platform --param-value=win

and only tests parametrized with platform=win will be executed.

like image 59
hoefling Avatar answered Nov 12 '22 16:11

hoefling


An alternative to the official answer by hoefling is to create a special marker with pytest-pilot and to apply it:

conftest.py:

from pytest_pilot import EasyMarker

mymark = EasyMarker('mymark', has_arg=False, mode='hard_filter')

test_so.py:

import pytest
from .conftest import mymark

@pytest.mark.parametrize('platform,configuration', (
        mymark.param('win', 'release'),
        pytest.param('win', 'debug')
))
def test_foo(platform, configuration):
    pass

You can now run pytest --mymark, it correctly runs only the test with the mark

test_so\test_so.py::test_foo[win-release] PASSED                         [ 50%]
test_so\test_so.py::test_foo[win-debug] SKIPPED                          [100%]

Of course it might not be relevant in all cases, as it requires code modification ; however for advanced filtering patterns, or if the filtering is here to stay and you wish to have some CLI shortcuts to perform it, it might be interesting. Note: I'm the author of this lib ;)

like image 35
smarie Avatar answered Nov 12 '22 14:11

smarie