How can I access the overall test result of a pytest test run during runtime?

Question

Dependent on the overall test result of a pytest test run I would like to execute conditional tear down. This means the access to the overall test result must happen after all tests have been executed but before the test runner has been left. How can I achieve this?

Answer 1

I could not find a suitable pytest hook to access the overall test result yet.

You don't need one; just collect the test results yourself. This is the blueprint I usually use when in need of accessing the test results in batch:

# conftest.py
import pytest


def pytest_sessionstart(session):
    session.results = dict()


@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()

    if result.when == 'call':
        item.session.results[item] = result

Now all test results are stored under session.results dict; example usage:

# conftest.py (continued)

def pytest_sessionfinish(session, exitstatus):
    print()
    print('run status code:', exitstatus)
    passed_amount = sum(1 for result in session.results.values() if result.passed)
    failed_amount = sum(1 for result in session.results.values() if result.failed)
    print(f'there are {passed_amount} passed and {failed_amount} failed tests')

Running the tests will yield:

$ pytest -sv
================================== test session starts ====================================
platform darwin -- Python 3.6.4, pytest-3.7.1, py-1.5.3, pluggy-0.7.1 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-51711988, inifile:
collected 3 items

test_spam.py::test_spam PASSED
test_spam.py::test_eggs PASSED
test_spam.py::test_fail FAILED
run status code: 1
there are 2 passed and 1 failed tests


======================================== FAILURES =========================================
_______________________________________ test_fail _________________________________________

    def test_fail():
>       assert False
E       assert False

test_spam.py:10: AssertionError
=========================== 1 failed, 2 passed in 0.05 seconds ============================

In case the overall pytest exit code (exitstatus) is sufficient info (info about # passed, # failed, etc. not required) use the following:

# conftest.py

def pytest_sessionfinish(session, exitstatus):
    print()
    print('run status code:', exitstatus)

Accessing the error

You can access the error details from the call.excinfo object:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()
    if result.outcome == "failed":
        exception = call.excinfo.value
        exception_class = call.excinfo.type
        exception_class_name = call.excinfo.typename
        exception_type_and_message_formatted = call.excinfo.exconly()
        exception_traceback = call.excinfo.traceback