How to detect when pytest test case failed?

Question

I am using pytest with selenium to automate a website. I want to take some screen shot only when a test case fails. I have previosly used TestNG and with TestNG it's quite east using the ITestListner. Do we have something like that in pytest.

I have tried to achieve this using the teardown_method() But this method is not getting executed when a test case fails.

import sys

from unittestzero import Assert
class TestPY:
    def setup_method(self, method):
        print("in setup method")
        print("executing " + method.__name__)

    def teardown_method(self, method):
        print(".....teardown")
        if sys.exc_info()[0]:
            test_method_name = method
            print test_method_name

    def test_failtest(self):
        Assert.fail("failed test")

teardown_method() get executed only when there are no fails

Answer 1

According to you post on stackoverflow, I can share that I is something on my mind, I hope it will help:wink: What you're trying to do is to handle standard AssertionError exception that can be raised by assert keyword or by any assertion method implemented in unittest.TestCase or maybe any custom assertion method that raises custom exception. There are 3 ways to do that:

1. Use try-except-finally construction. Some basic example:

   try:
       Assert.fail("failed test")
   except AssertionError:
       get_screenshot()
       raise
   

2. Or use with statement, as context manager:

   class TestHandler:
       def __enter__(self):
           #  maybe some set up is expected before assertion method call
           pass
   
       def __exit__(self, exc_type, exc_val, exc_tb):
           #  catch whether exception was raised
           if isinstance(exc_val, AssertionError):
               get_screenshot()
   
   
   with TestHandler():
       Assert.fail("failed test")
   

   here you can dive deeper on how to play with it

3. The last one, in my opinion, is the most elegant approach. Using decorators. With this decorator you can decorate any testing method:

   def decorator_screenshot(func):
       def wrapper(*args, **kwargs):
           try:
              func(*args, **kwargs)
           except AssertionError:
              get_screenshot()
              raise
       return wrapper
   
   
   @decorator_screenshot
   def test_something():
       Assert.fail("failed test")
   

Answer 2

After some struggle, eventually this worked for me.

In conftest.py:

@pytest.hookimpl(hookwrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    rep = outcome.get_result()
    setattr(item, "rep_" + rep.when, rep)
    return rep

And, in your code, in a fixture (e.g., in a teardown fixture for tests) use it like so:

def tear_down(request):
    method_name = request.node.name
    if request.node.rep_call.failed:
        print('test {} failed :('.format(method_name))
        # do more stuff like take a selenium screenshot

Note that "request" is a fixture "funcarg" that pytest provides in the context of your tests. You don't have to define it yourself.

Sources: pytest examples and thread on (not) making this easier.

Answer 3

This is how we do it , note __multicall__ has very less documentation and I remember reading __multicall__ is going to be deprecated, please use this with a pinch of salt and experiment with replacing __multicall__ with 'item, call' as per the examples.

def pytest_runtest_makereport(__multicall__):
    report = __multicall__.execute()

    if report.when == 'call':
        xfail = hasattr(report, 'wasxfail')
        if (report.skipped and xfail) or (report.failed and not xfail):

           try:
              screenshot = APP_DRIVER.take_screen_shot(format="base64")


           except Exception as e:
              LOG.debug("Error saving screenshot !!")
              LOG.debug(e)

    return report