With Python unittest, how do I create and use a "callable object that returns a test suite"?

Question

I'm learning Python and have been trying to understand more about the details of Python's unittest module. The documentation includes the following:

For the ease of running tests, as we will see later, it is a good idea to provide in each test module a callable object that returns a pre-built test suite:

def suite():
    suite = unittest.TestSuite()
    suite.addTest(WidgetTestCase('testDefaultSize'))
    suite.addTest(WidgetTestCase('testResize'))
    return suite

As far as I can tell, the purpose of doing this is not explained. In addition, I was unable to figure out how one would use such a method. I tried several things without success (aside from learning about the error messages I got):

import unittest

def average(values):
    return sum(values) / len(values)

class MyTestCase(unittest.TestCase):
    def testFoo(self):
        self.assertEqual(average([10,100]),55)

    def testBar(self):
        self.assertEqual(average([11]),11)

    def testBaz(self):
        self.assertEqual(average([20,20]),20)

    def suite():
        suite = unittest.TestSuite()
        suite.addTest(MyTestCase('testFoo'))
        suite.addTest(MyTestCase('testBar'))
        suite.addTest(MyTestCase('testBaz'))
        return suite

if __name__ == '__main__':
    # s = MyTestCase.suite()
    # TypeError: unbound method suite() must be called 
    # with MyTestCase instance as first argument

    # s = MyTestCase.suite(MyTestCase())
    # ValueError: no such test method in <class '__main__.MyTestCase'>: runTest

    # s = MyTestCase.suite(MyTestCase('testFoo'))
    # TypeError: suite() takes no arguments (1 given)

The following "worked" but seems awkward and it required that I change the method signature of suite() to 'def suite(self):'.

s = MyTestCase('testFoo').suite()
unittest.TextTestRunner().run(s)

Answer 1

The very first error message you got is meaningful, and explains a lot.

print MyTestCase.suite # <unbound method MyTestCase.suite>

Unbound. It means that you cannot call it unless you bind it to an instance. It's actually the same for MyTestCase.run:

print MyTestCase.run # <unbound method MyTestCase.run>

Maybe for now you don't understand why you can't call suite, but please leave it aside for now. Would you try to call run on the class, like above? Something like:

MyTestCase.run() # ?

Probably not, right? It does not make sense to write this, and it will not work, because run is an instance method, and needs a self instance to work on. Well it appears that Python "understands" suite in the same way it understands run, as an unbound method.

Let's see why:

If you try to put the suite method out of the class scope, and define it as a global function, it just works:

import unittest

def average(values):
    return sum(values) / len(values)

class MyTestCase(unittest.TestCase):
    def testFoo(self):
        self.assertEqual(average([10,100]),55)

    def testBar(self):
        self.assertEqual(average([11]),11)

    def testBaz(self):
        self.assertEqual(average([20,20]),20)

def suite():
    suite = unittest.TestSuite()
    suite.addTest(MyTestCase('testFoo'))
    suite.addTest(MyTestCase('testBar'))
    suite.addTest(MyTestCase('testBaz'))
    return suite

print suite() # <unittest.TestSuite tests=[<__main__.MyTestCase testMethod=testFoo>, <__main__.MyTestCase testMethod=testBar>, <__main__.MyTestCase testMethod=testBaz>]>

But you don't want it out of the class scope, because you want to call MyTestCase.suite()

You probably thought that since suite was sort of "static", or instance-independent, it did not make sense to put a self argument, did you? It's right.

But if you define a method inside a Python class, Python will expect that method to have a self argument as a first argument. Just omitting the self argument does not make your method static automatically. When you want to define a "static" method, you have to use the staticmethod decorator:

@staticmethod
def suite():
    suite = unittest.TestSuite()
    suite.addTest(MyTestCase('testFoo'))
    suite.addTest(MyTestCase('testBar'))
    suite.addTest(MyTestCase('testBaz'))
    return suite

This way Python does not consider MyTestCase as an instance method but as a function:

print MyTestCase.suite # <function suite at 0x...>

And of course now you can call MyTestCase.suite(), and that will work as expected.

if __name__ == '__main__':
    s = MyTestCase.suite()
    unittest.TextTestRunner().run(s) # Ran 3 tests in 0.000s, OK