Outputting data from unit test in Python

Question

We use the logging module for this.

For example:

import logging
class SomeTest( unittest.TestCase ):
    def testSomething( self ):
        log= logging.getLogger( "SomeTest.testSomething" )
        log.debug( "this= %r", self.this )
        log.debug( "that= %r", self.that )
        # etc.
        self.assertEquals( 3.14, pi )

if __name__ == "__main__":
    logging.basicConfig( stream=sys.stderr )
    logging.getLogger( "SomeTest.testSomething" ).setLevel( logging.DEBUG )
    unittest.main()

That allows us to turn on debugging for specific tests which we know are failing and for which we want additional debugging information.

My preferred method, however, isn't to spend a lot of time on debugging, but spend it writing more fine-grained tests to expose the problem.

In Python 2.7 you could use an additional parameter, msg, to add information to the error message like this:

self.assertEqual(f.bar(t2), 2, msg='{0}, {1}'.format(t1, t2))

The official documentation is here.

You can use simple print statements, or any other way of writing to standard output. You can also invoke the Python debugger anywhere in your tests.

If you use nose to run your tests (which I recommend), it will collect the standard output for each test and only show it to you if the test failed, so you don't have to live with the cluttered output when the tests pass.

nose also has switches to automatically show variables mentioned in asserts, or to invoke the debugger on failed tests. For example, -s (--nocapture) prevents the capture of standard output.

I don't think this is quite what you're looking for. There's no way to display variable values that don't fail, but this may help you get closer to outputting the results the way you want.

You can use the TestResult object returned by the TestRunner.run() for results analysis and processing. Particularly, TestResult.errors and TestResult.failures

About the TestResults Object:

http://docs.python.org/library/unittest.html#id3

And some code to point you in the right direction:

>>> import random
>>> import unittest
>>>
>>> class TestSequenceFunctions(unittest.TestCase):
...     def setUp(self):
...         self.seq = range(5)
...     def testshuffle(self):
...         # make sure the shuffled sequence does not lose any elements
...         random.shuffle(self.seq)
...         self.seq.sort()
...         self.assertEqual(self.seq, range(10))
...     def testchoice(self):
...         element = random.choice(self.seq)
...         error_test = 1/0
...         self.assert_(element in self.seq)
...     def testsample(self):
...         self.assertRaises(ValueError, random.sample, self.seq, 20)
...         for element in random.sample(self.seq, 5):
...             self.assert_(element in self.seq)
...
>>> suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
>>> testResult = unittest.TextTestRunner(verbosity=2).run(suite)
testchoice (__main__.TestSequenceFunctions) ... ERROR
testsample (__main__.TestSequenceFunctions) ... ok
testshuffle (__main__.TestSequenceFunctions) ... FAIL

======================================================================
ERROR: testchoice (__main__.TestSequenceFunctions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<stdin>", line 11, in testchoice
ZeroDivisionError: integer division or modulo by zero

======================================================================
FAIL: testshuffle (__main__.TestSequenceFunctions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<stdin>", line 8, in testshuffle
AssertionError: [0, 1, 2, 3, 4] != [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

----------------------------------------------------------------------
Ran 3 tests in 0.031s

FAILED (failures=1, errors=1)
>>>
>>> testResult.errors
[(<__main__.TestSequenceFunctions testMethod=testchoice>, 'Traceback (most recent call last):\n  File "<stdin>"
, line 11, in testchoice\nZeroDivisionError: integer division or modulo by zero\n')]
>>>
>>> testResult.failures
[(<__main__.TestSequenceFunctions testMethod=testshuffle>, 'Traceback (most recent call last):\n  File "<stdin>
", line 8, in testshuffle\nAssertionError: [0, 1, 2, 3, 4] != [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n')]
>>>