How to mark one function not a test for pytest?

Question

I'm using pytest to test some code based on TensorFlow.

A TestCase is defined for simplicity like:

class TestCase(tf.test.TestCase):
    # ...

The problem is tf.test.TestCase provide a useful function self.test_session(), which was treated as a test method in pytest since its name starts with test_.

The result pytest report more succeed tests than test methods I defined due to test_session() methods.

I use the following code to skip test_session:

class TestCase(tf.test.TestCase):
    @pytest.mark.skip
    @contextmanager
    def test_session(self):
        with super().test_session() as sess:
            yield sess

However there would be some "s" in test report indicating there are some skip tests.

Is there anyway I can mark one exact method not a test method without changing pytest test discovery rules globally?

Answer 1

Filter out false positives after the test items are collected: create a conftest.py in your tests directory with the custom post-collection hook:

# conftest.py
def pytest_collection_modifyitems(session, config, items):
    items[:] = [item for item in items if item.name != 'test_session']

pytest will still collect the test_session methods (you will notice that in the pytest report line collected n tests), but not execute them as tests and not consider them anywhere in the test run.

---

Related: fix for unittest-style tests

Check out this answer.

Answer 2

You can set __test__ = False, either directly, or by writing a simple decorator. The latter should behave similarly to Nose's nottest decorator.

def nottest(obj):
    obj.__test__ = False
    return obj

class TestMyTest:

    def test_should_not_collect_1(self):
        assert False
    test_should_not_collect_1.__test__ = False

    @nottest
    def test_should_not_collect_2(self):
        assert False

    def test_should_collect(self):
        assert True


def test_should_not_collect_1():
    assert False
test_should_not_collect_1.__test__ = False

@nottest
def test_should_not_collect_2():
    assert False


def test_should_collect():
    assert True

When running pytest, this only runs the methods which are not marked:

$ pytest test.py -v
====================================== test session starts ======================================
platform darwin -- Python 3.9.1, pytest-7.0.1, pluggy-1.0.0 -- /Users/lucaswiman/.pyenv/versions/3.9.1/bin/python3.9
cachedir: .pytest_cache
rootdir: /private/tmp
plugins: anyio-2.2.0
collected 2 items                                                                               

test.py::TestMyTest::test_should_collect PASSED                                           [ 50%]
test.py::test_should_collect PASSED                                                       [100%]

======================================= 2 passed in 0.04s =======================================

This behavior is documented here:

Since Pytest 2.6, users can prevent pytest from discovering classes that start with Test by setting a boolean __test__ attribute to False.