Unit Testing Interfaces in Python

Question

I am currently learning python in preperation for a class over the summer and have gotten started by implementing different types of heaps and priority based data structures.

I began to write a unit test suite for the project but ran into difficulties into creating a generic unit test that only tests the interface and is oblivious of the actual implementation.

I am wondering if it is possible to do something like this..

suite = HeapTestSuite(BinaryHeap())
suite.run()
suite = HeapTestSuite(BinomialHeap())
suite.run()

What I am currently doing just feels... wrong (multiple inheritance? ACK!)..

class TestHeap:

    def reset_heap(self):
        self.heap = None

    def test_insert(self):
        self.reset_heap()
        #test that insert doesnt throw an exception...
        for x in self.inseq:
            self.heap.insert(x)


    def test_delete(self):
        #assert we get the first value we put in
        self.reset_heap()
        self.heap.insert(5)
        self.assertEquals(5, self.heap.delete_min())

        #harder test. put in sequence in and check that it comes out right
        self.reset_heap()
        for x in self.inseq:
            self.heap.insert(x)

        for x in xrange(len(self.inseq)):
            val = self.heap.delete_min()
            self.assertEquals(val, x)

class BinaryHeapTest(TestHeap, unittest.TestCase):
    def setUp(self):
        self.inseq = range(99, -1, -1)
        self.heap = BinaryHeap()

    def reset_heap(self):
        self.heap = BinaryHeap()

class BinomialHeapTest(TestHeap, unittest.TestCase):
    def setUp(self):
        self.inseq = range(99, -1, -1)
        self.heap = BinomialHeap()

    def reset_heap(self):
        self.heap = BinomialHeap()


if __name__ == '__main__':
    unittest.main()

Answer 1

I personally like nose test generation more for this sort of thing. I'd then write it like:

# They happen to all be simple callable factories, if they weren't you could put
# a function in here:
make_heaps = [BinaryHeap, BinomialHeap]

def test_heaps():
    for make_heap in make_heaps:
        for checker in checkers: # we'll set checkers later
            yield checker, make_heap

def check_insert(make_heap):
    heap = make_heap()
    for x in range(99, -1, -1):
        heap.insert(x)

# def check_delete_min etc.

checkers = [
    value
    for name, value in sorted(globals().items())
    if name.startswith('check_')]

Answer 2

Why not just use an alias for the class you want to test? You can write your test class referring to a fake HeapImpl class, and then assign a specific implementation to it before each test run:

class TestHeap(unittest.TestCase):
    def setUp(self):
        self.heap = HeapImpl()
    #test cases go here

if __name__ == '__main__'
    suite = unittest.TestLoader().loadTestsFromTestCase(TestHeap)
    heaps = [BinaryHeap, BinomialHeap]
    for heap in heaps:
        HeapImpl = heap
        unittest.TextTestRunner().run(suite)

As long as they comply with the interface you're using in the test suite, this should work fine. Also, you can easily test as many implementations as you want, just add them to the heaps list.

Answer 3

I don't think the above pattern is terrible, but multiple inheritance is certainly not idea.

I guess the reason you can't just have TestHeap be a subclass of TestCase is because it will automatically be picked up and run as test, not knowing that it needs to be subclassed.

I've gotten around this problem two other ways:

1. Rather than adding test_ functions, have write methods that don't automatically get picked up, then add test() to each of your subclasses. Obviously not ideal.
2. Rewrote unittest to not suck, allowing the option of setting __test__ = False to the base class. (See Testify)