Does pytest have anything like google test's non-fatal EXPECT_* behavior?

Question

I'm more familiar with the google test framework and know about the primary behavior pair they support about ASSERT_* vs EXPECT_* which are the fatal and non-fatal assert modes.

From the documentation:

The assertions come in pairs that test the same thing but have different effects on the current function. ASSERT_* versions generate fatal failures when they fail, and abort the current function. EXPECT_* versions generate nonfatal failures, which don't abort the current function. Usually EXPECT_* are preferred, as they allow more than one failures to be reported in a test. However, you should use ASSERT_* if it doesn't make sense to continue when the assertion in question fails.

Question: does pytest also have a non fatal assert flavor or mode I can enable?

It's nice to allow a full range of tests to maximally execute to get the richest failure history rather than abort at the first failure and potentially hide subsequent failures that have to be discovered piecewise by running multiple instances of the test application.

Answer 1

I use pytest-assume for non-fatal assertions. It does the job pretty well.

Installation

As usual,

$ pip install pytest-assume

Usage example

import pytest


def test_spam():
    pytest.assume(True)
    pytest.assume(False)

    a, b = True, False
    pytest.assume(a == b)

    pytest.assume(1 == 0)
    pytest.assume(1 < 0)

    pytest.assume('')
    pytest.assume([])
    pytest.assume({})

If you feel writing pytest.assume is a bit too much, just alias the import:

import pytest.assume as expect


def test_spam():
    expect(True)
    ...

Running the above test yields:

$ pytest -v
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.6.0, py-1.5.3, pluggy-0.6.0 -- /data/gentoo64-prefix/u0_a82/projects/stackoverflow/so-50630845
cachedir: .pytest_cache
rootdir: /data/gentoo64-prefix/u0_a82/projects/stackoverflow/so-50630845, inifile:
plugins: assume-1.2
collecting ... collected 1 item

test_spam.py::test_spam FAILED                                            [100%]

=================================== FAILURES ===================================
__________________________________ test_spam ___________________________________
test_spam.py:6: AssumptionFailure
        pytest.assume(False)


test_spam.py:9: AssumptionFailure
        pytest.assume(a == b)


test_spam.py:11: AssumptionFailure
        pytest.assume(1 == 0)


test_spam.py:12: AssumptionFailure
        pytest.assume(1 < 0)


test_spam.py:13: AssumptionFailure
        pytest.assume('')


test_spam.py:14: AssumptionFailure
        pytest.assume([])


test_spam.py:14: AssumptionFailure
        pytest.assume([])


test_spam.py:15: AssumptionFailure
        pytest.assume({})

------------------------------------------------------------
Failed Assumptions: 7
=========================== 1 failed in 0.18 seconds ===========================

Answer 2

No, there is no feature like that in pytest. The most popular approach is to use regular assert statements, which fail the test immediately if the expression is falsey.

It's nice to allow a full range of tests to maximally execute to get the richest failure history rather than abort at the first failure and potentially hide subsequent failures that have to be discovered piecewise by running multiple instances of the test application.

Opinions differ on whether this is nice or not. In the open source Python community, at least, the popular approach is: every potential "subsequent failure that is discovered piecewise" would be written in its own separate test. More tests, smaller tests, that (ideally) only assert on one thing.

You could easily recreate the EXPECT_* thing by appending to a list of errors and then asserting the list is empty at the end of the test, but there is no support directly in pytest for such a feature.