pytest: Reusable tests for different implementations of the same interface

Question

Imagine I have implemented a utility (maybe a class) called Bar in a module foo, and have written the following tests for it.

test_foo.py:

from foo import Bar as Implementation
from pytest import mark

@mark.parametrize(<args>, <test data set 1>)
def test_one(<args>):
    <do something with Implementation and args>

@mark.parametrize(<args>, <test data set 2>)
def test_two(<args>):
    <do something else with Implementation and args>

<more such tests>

Now imagine that, in the future I expect different implementations of the same interface to be written. I would like those implementations to be able to reuse the tests that were written for the above test suite: The only things that need to change are

1. The import of the Implementation
2. <test data set 1>, <test data set 2> etc.

So I am looking for a way to write the above tests in a reusable way, that would allow authors of new implementations of the interface to be able to use the tests by injecting the implementation and the test data into them, without having to modify the file containing the original specification of the tests.

What would be a good, idiomatic way of doing this in pytest?

====================================================================

====================================================================

Here is a unittest version that (isn't pretty but) works.

define_tests.py:

# Single, reusable definition of tests for the interface. Authors of
# new implementations of the interface merely have to provide the test
# data, as class attributes of a class which inherits
# unittest.TestCase AND this class.
class TheTests():

    def test_foo(self):
        # Faking pytest.mark.parametrize by looping
        for args, in_, out in self.test_foo_data:
            self.assertEqual(self.Implementation(*args).foo(in_),
                             out)

    def test_bar(self):
        # Faking pytest.mark.parametrize by looping
        for args, in_, out in self.test_bar_data:
            self.assertEqual(self.Implementation(*args).bar(in_),
                             out)

v1.py:

# One implementation of the interface
class Implementation:

    def __init__(self, a,b):
        self.n = a+b

    def foo(self, n):
        return self.n + n

    def bar(self, n):
        return self.n - n

v1_test.py:

# Test for one implementation of the interface
from v1 import Implementation
from define_tests import TheTests
from unittest import TestCase

# Hook into testing framework by inheriting unittest.TestCase and reuse
# the tests which *each and every* implementation of the interface must
# pass, by inheritance from define_tests.TheTests
class FooTests(TestCase, TheTests):

    Implementation = Implementation

    test_foo_data = (((1,2), 3,  6),
                     ((4,5), 6, 15))

    test_bar_data = (((1,2), 3,  0),
                     ((4,5), 6,  3))

Anybody (even a client of the library) writing another implementation of this interface

• can reuse the set of tests defined in define_tests.py
• inject own test data into the tests
• without modifying any of the original files

Answer 1

This is a great use case for parametrized test fixtures.

Your code could look something like this:

from foo import Bar, Baz

@pytest.fixture(params=[Bar, Baz])
def Implementation(request):
    return request.param

def test_one(Implementation):
    assert Implementation().frobnicate()

This would have test_one run twice: once where Implementation=Bar and once where Implementation=Baz.

Note that since Implementation is just a fixture, you can change its scope, or do more setup (maybe instantiate the class, maybe configure it somehow).

If used with the pytest.mark.parametrize decorator, pytest will generate all the permutations. For example, assuming the code above, and this code here:

@pytest.mark.parametrize('thing', [1, 2])
def test_two(Implementation, thing):
    assert Implementation(thing).foo == thing

test_two will run four times, with the following configurations:

• Implementation=Bar, thing=1
• Implementation=Bar, thing=2
• Implementation=Baz, thing=1
• Implementation=Baz, thing=2