How do I use unittest.TestResult?

Question

I've only been using unittest for a short time. I am using Jython 2.7.10 "final release"

In the Python 2.7 docs explaining TestResult it says:

The following methods of the TestResult class are used to maintain the internal data structures, and may be extended in subclasses to support additional reporting requirements. This is particularly useful in building tools which support interactive reporting while tests are being run.

startTest(test) ... stopTest(test) ... startTestRun() ... stopTestRun()¶

That's what I want to do... but I can't work out how you use TestResult. Here's an SSCCE...

import unittest

class TestResultX( unittest.TestResult ):
    def startTest( self, test ):
        print( '# blip')
        unittest.TestResult.startTest( self, test )
    def stopTest( self, test ):
        print( '# blop')
        unittest.TestResult.stopTest( self, test )
    def startTestRun( self ):
        print( '# blep')
        unittest.TestResult.startTestRun( self )
    def stopTestRun( self ):
        print( '# blap')
        unittest.TestResult.stopTestRun( self )

class TestCaseX( unittest.TestCase ):
    def test_nonsense(self):
        print( '# wotcha' )
        self.assertTrue( False )

    def run( self, test_result=None ):
        print( '# spoons starting...')

        test_result = TestResultX()
        unittest.TestCase.run( self, test_result )

        print( '# ...spoons ended, tr %s' % ( test_result,  ) )

unittest.main()

Results in:

# spoons starting...

----------------------------------------------------------------------
Ran 0 tests in 0.015s

OK
# blip
# wotcha
# blop
# ...spoons ended, tr <__main__.TestResultX run=1 errors=0 failures=1>

Questions:

• Why does it say 0 tests?
• Why are blep and blap (start and end of run) not printed?

On a more general note:

1. Can someone possibly point to a good tutorial/book explaining "proper use"/"good practice" when it comes to TestResult, TestRunner, TestLoader, etc. I got "TDD with Python", but it doesn't seem to explain any of this.

2. Can someone possibly tell me why unittest2 often seems to be used instead of unittest?

addendum

Following Omar Diab's efforts at looking at the source code I tried this:

def run( self, *args, **kvargs ):
    result = self.defaultTestResult()
    startTestRun = getattr(result, 'startTestRun', None)
    logger.info( '# calling superclass run... startTestRun? %s' % ( startTestRun, ))
    unittest.TestCase.run( self, *args, **kvargs  )
    logger.info( '# ... superclass run ended')

Unfortunately each test_XXX method then gave:

# calling superclass run... startTestRun? <bound method TestResult.startTestRun of <unittest.result.TestResult run=0 errors=0 failures=0>>

setUp for test_that_stuff_happened (__main__.xx_FT)

tearDown for test_that_stuff_happened (__main__.xx_FT)
end tearDown...
. # ... superclass run ended

Answer 1

Wow, no responses. I'm surprised.

This is a hack which most people could no doubt work out for themselves, if you want stuff to happen at the start of the run and end of the run:

Subclass TestCase, as per:

def setUp( self ):
    if not hasattr( unittest.TestCase, 'app' ):
        unittest.TestCase.app = MyApp()
        def shutdown_func():
            pass # do any end-of-run stuff here
        atexit.register( shutdown_func )
        pass # do any start-of-run stuff here
    self.app = unittest.TestCase.app

Then make all your TestCases subclass from this one...

The point being, if you want this to happen, that your app is only constructed once. Handling the responsibility of ensuring that it is "pristine" for each successive setUp is up to you of course. Obviously you could use setUpClass instead but then you don't have access to the TestCase instance.

Answer 2

I am having the same problem, so I took a look at the source code.

Inspecting unittest.TextTestRunner and unittest.TestCase, it looks like startTestRun() and stopTestRun() get called manually. In unittest.TextTestRunner, it works like so:

def run(self, test):
    # ...
    startTestRun = getattr(result, 'startTestRun', None)
    if startTestRun is not None:
        startTestRun()
    # ...

and in your case, unittest.TestCase, it is like so:

def run(self, result=None):
    orig_result = result
    if result is None:
        result = self.defaultTestResult()
        startTestRun = getattr(result, 'startTestRun', None)
        if startTestRun is not None:
            startTestRun()
    # ...

So it looks like startTestRun only actually gets called by TestCase.run() if you don't pass in result. You're passing in result, so it's not happening.

This seems like a bug to me! But basically it means you can extend TestCase or TestSuite, reimplement the run method, and then call those methods by hand; or just call them outside of those respective run methods.

Hope this helps!