Integration testing with Python unittest: how to improve granularity?

Question

I have a single end-to-end Selenium integration test which consists of several steps with each step setting certain state and passing the data over to the next step.

Test scenario:

1. Create item.
2. Assign item to group.
3. Approve item.
...
4. Publish item.

The steps are not atomic and depend on each other. The test class extends from unittest.TestCase and contains a single test_xyz() method which wraps the methods for each step.

Currently, the final report only includes the pass/fail status for the wrapper test_xyz() method. I would like to have each step being treated as a 'test step' and have a separate entry in the test report.

It feels wrong to convert all the steps into test_step1(), test_step2() etc and then enforcing the execution order and data exchange since the steps are not self-contained tests.

Is there a way to mark certain methods as test steps so that they appear as separate entries in the test report?

Basically, all I want is improving the visual reporting so that it is more obvious from the report at which step the entire integration test failed. Is this achievable with unittest? What would be the best practice in organizing the code so that it fits into the unittest paradigm?

Not sure unittest is the best tool for integration tests, since this is not strictly speaking unit testing. Would switching to something like pytest make more sense for integration testing while still allowing 'free' HTML report generation? (I am currently using nose to run the test and to generate the HTML report with the corresponding nose plugin.)

Answer 1

pytest

The most flexible option turned out to be switching to pytest with @pytest.mark.incremental and pytest-html for HTML report generation.

Also planning to look into the pytest-selenium plugin as a possibly cleaner way to organize the selenium fixtures.

The following article proved useful in understanding how to maintain shared state between tests steps: http://computableverse.com/blog/pytest-sharing-class-fixtures

unittest

And if you prefer to stick with unittest, then for simple use cases like iteration you can use unittest.subTest as demonstrated in the example below:

#!/usr/bin/env python3
import unittest

import requests

HOST='http://httpbin.org'

class TestEndpoints(unittest.TestCase):

    def test_endpoints(self):

        for endpoint in ('/headers',
                         '/ip',
                         '/user-agent',
                         '/response-headers',
                         '/html',
                         ):
            with self.subTest(path=endpoint):

                url = HOST + endpoint
                response = requests.get(url)
                print(url, response.status_code)
                self.assertEqual(response.status_code, requests.codes.ok, msg=f'expected status code {requests.codes.ok}')


if __name__ == '__main__':
    unittest.main()