Soft Asserts in unittest python

Question

Currently I have a test case that loops through a dictionary of dictionaries, each of these containing a separate value that I want to test on a web page (I am using Selenium Webdriver, though that is not necessarily relevant to the question). This test case essentially will check that all possible purchase paths for a product are working correctly, which comes out to be about 200 different paths. I want to keep the code simple and not make a test for each one. Below is an example of what I have done to shorten the code:

self.templates = {"sales": self.sales", ...)
self.template_keys = ["sales",....]
self.sales - {"locator1": "<locataor info>, ...)
.... <more dictionaries>

for key in self.template_keys:
    for template in self.templates[key]:
        <do purchase path in selenium webdriver>
        assert end_url == expected_end_url # Would like failure to not end test

My question is how to get the assert in the nested for loop to not fatally fail and move on to the next step. I have read this stackoverflow article and it pretty much seems like he decided to do his own thing without giving any information on what he did. I know that this is an anti-pattern, but it is not worth my time to write all of them out separately. I am wondering if anyone has a good solution, something that works like soft asserts in Groovy.

Answer 1

Collect a report of multiple failures: Replace the assert by an if and create a descriptor for each failure in its body. Collect these descriptors in a list (initially empty: failures = []):

if end_url != expected_end_url:
    failures.append(end_url + ' != ' + expected_end_url)

In the end, assert the list is empty and use it as the error message if it is not:

assert(failures == [], str(failures))

Much more readable than catching exceptions -- and very flexible, too.

Answer 2

Today I have exactly the same desire for soft assertions, something I occasionally used at my last job with Java and TestNG. I'm a little surprised nothing like this is built-in to pytest. But I found there are (at least) two Python libraries for this.

The first is softest: https://pypi.org/project/softest/

The second is Python-Delayed-Assert: https://github.com/pr4bh4sh/python-delayed-assert

I haven't personally used either of them yet, but looking over the examples, it looks like they solve the same problem in essentially the same way.