Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does pytest support "default" markers?

Tags:

python

pytest

I am using pytest to test python models for embedded systems. Features to be tested vary by platform. ( I'm using 'platform' in this context to mean an embedded system type, not an OS type).

The most straightforward way to organize my tests would be to allocate them to directories based on platform type.

/platform1
/platform2
/etc.

pytest /platform1

This quickly became hard to support as many features overlap across platforms. I've since moved my tests into a single directory, with tests for each functional area assigned to a single filename (test_functionalityA.py, for example). I then use pytest markers to indicate which tests within a file apply to a given platform.

@pytest.mark.all_platforms
def test_some_functionalityA1():
    ...

@pytest.mark.platform1
@pytest.mark.platform2
def test_some_functionlityA2():
    ...

While I would love to get 'conftest' to automatically detect the platform type and only run the appropriate tests, I've resigned myself to specifying which tests to run on the command line.

pytest -m "(platform1 or all_platforms)"

The Question: (finally!)

Is there a way to simplify things and have pytest run all unmarked tests by default and additionally all tests passed via '-m' on the command-line?

For example: pytest -m "platform1"

would run tests marked @pytest.mark.platform1 as well as all tests marked @pytest.mark.all_platforms or even all tests with no @pytest.mark at all?

Given the large amount of shared functionality, being able to drop the @pytest.mark.all_platforms line would be a big help.

like image 302
JS. Avatar asked Jun 06 '12 00:06

JS.


People also ask

How do you use pytest markers?

To use markers, we have to import pytest module in the test file. We can define our own marker names to the tests and run the tests having those marker names. -m <markername> represents the marker name of the tests to be executed. Update our test files test_compare.py and test_square.py with the following code.

Can we assign multiple markers in pytest?

Multiple custom markers can be registered, by defining each one in its own line, as shown in above example. You can ask which markers exist for your test suite - the list includes our just defined webtest and slow markers: $ pytest --markers @pytest. mark.

Which command can be used to show all available fixtures?

Additionally, if you wish to display a list of fixtures for each test, try the --fixtures-per-test flag.


2 Answers

Let's tackle the full problem. I think you can put a conftest.py file along with your tests and it will take care to skip all non-matching tests (non-marked tests will always match and thus never get skipped). Here i am using sys.platform but i am sure you have a different way to compute your platform value.

# content of conftest.py
#
import sys
import pytest

ALL = set("osx linux2 win32".split())

def pytest_runtest_setup(item):
    if isinstance(item, item.Function):
        plat = sys.platform
        if not hasattr(item.obj, plat):
            if ALL.intersection(set(item.obj.__dict__)):
                pytest.skip("cannot run on platform %s" %(plat))

With this you can mark your tests like this::

# content of test_plat.py

import pytest

@pytest.mark.osx
def test_if_apple_is_evil():
    pass

@pytest.mark.linux2
def test_if_linux_works():
    pass

@pytest.mark.win32
def test_if_win32_crashes():
    pass

def test_runs_everywhere_yay():
    pass

and if you run with::

$ py.test -rs

then you can run it and will see at least two test skipped and always at least one test executed::

then you will see two test skipped and two executed tests as expected::

$ py.test -rs # this option reports skip reasons
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1
collecting ... collected 4 items

test_plat.py s.s.
========================= short test summary info ==========================
SKIP [2] /home/hpk/tmp/doc-exec-222/conftest.py:12: cannot run on platform linux2

=================== 2 passed, 2 skipped in 0.01 seconds ====================

Note that if you specify a platform via the marker-command line option like this::

$ py.test -m linux2
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1
collecting ... collected 4 items

test_plat.py .

=================== 3 tests deselected by "-m 'linux2'" ====================
================== 1 passed, 3 deselected in 0.01 seconds ==================

then the unmarked-tests will not be run. It is thus a way to restrict the run to the specific tests.

like image 178
hpk42 Avatar answered Oct 21 '22 07:10

hpk42


Late to the party, but I just solved a similar problem by adding a default marker to all unmarked tests.

As a direct answer to the Question: you can have unmarked tests always run, and include marked test only as specified via the -m option, by adding the following to the conftest.py

def pytest_collection_modifyitems(items, config):
    # add `always_run` marker to all unmarked items
    for item in items:
        if not any(item.iter_markers()):
            item.add_marker("always_run")
    # Ensure the `always_run` marker is always selected for
    markexpr = config.getoption("markexpr", 'False')
    config.option.markexpr = f"always_run or ({markexpr})"
like image 28
Nat Avatar answered Oct 21 '22 05:10

Nat