Show progress while running python unittest?

Question

I have a very large TestSuite that I run with TextTestRunner from the python unittest framework. Unfortunately I have no idea how many tests are already done while the test are running.

Basically I'd like to convert this output:

test_choice (__main__.TestSequenceFunctions) ... ok
test_sample (__main__.TestSequenceFunctions) ... ok
test_shuffle (__main__.TestSequenceFunctions) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.110s

OK

to

[1/3] test_choice (__main__.TestSequenceFunctions) ... ok
[2/3] test_sample (__main__.TestSequenceFunctions) ... ok
[3/3] test_shuffle (__main__.TestSequenceFunctions) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.110s

OK

Do I have to subclass TextTestRunner to achieve this and if yes, how?

Note: I'm aware of nose and the available plugins, but it does not quite fit in my application and I'd like to avoid the dependency.

EDIT Why I'd like to avoid nose:

My Application is basically an additional framework for the tests. It select the correct test cases, provides library functions for them and executes the tests multiple times for long term testing. (the tests run against an external machine)

So here is how I run my tests right now:

# do all sort of preperations
[...]
test_suite = TestSuite()

repeatitions = 100
tests = get_tests()
for i in range(0, repeatitions):
    test_suite.addTests(tests)
TextTestRunner(verbosity=2).run(test_suite)

My problem with nose is, that it is designed to discover the tests from the filesystem and I was unable to find a clear documentation how to run it on a particular TestSuite directly from python.

Answer 1

Late to the party, but hopefully this will help anyone else who, like me, came here looking for an answer to this same problem:

Here's one way that you could subclass TextTestRunner and TextTestResult to achieve the desired result:

import unittest
import unittest.runner
import itertools
import collections

class CustomTextTestResult(unittest.runner.TextTestResult):
    """Extension of TextTestResult to support numbering test cases"""

    def __init__(self, stream, descriptions, verbosity):
        """Initializes the test number generator, then calls super impl"""

        self.test_numbers = itertools.count(1)

        return super(CustomTextTestResult, self).__init__(stream, descriptions, verbosity)

    def startTest(self, test):
        """Writes the test number to the stream if showAll is set, then calls super impl"""

        if self.showAll:
            progress = '[{0}/{1}] '.format(next(self.test_numbers), self.test_case_count)
            self.stream.write(progress)

            # Also store the progress in the test itself, so that if it errors,
            # it can be written to the exception information by our overridden
            # _exec_info_to_string method:
            test.progress_index = progress

        return super(CustomTextTestResult, self).startTest(test)

    def _exc_info_to_string(self, err, test):
        """Gets an exception info string from super, and prepends 'Test Number' line"""

        info = super(CustomTextTestResult, self)._exc_info_to_string(err, test)

        if self.showAll:
            info = 'Test number: {index}\n{info}'.format(
                index=test.progress_index,
                info=info
            )

        return info


class CustomTextTestRunner(unittest.runner.TextTestRunner):
    """Extension of TextTestRunner to support numbering test cases"""

    resultclass = CustomTextTestResult

    def run(self, test):
        """Stores the total count of test cases, then calls super impl"""

        self.test_case_count = test.countTestCases()
        return super(CustomTextTestRunner, self).run(test)

    def _makeResult(self):
        """Creates and returns a result instance that knows the count of test cases"""

        result = super(CustomTextTestRunner, self)._makeResult()
        result.test_case_count = self.test_case_count
        return result


class TestSequenceFunctions(unittest.TestCase):
    """Dummy test case to illustrate usage"""

    fail_1 = 0
    fail_2 = 0

    def test_choice(self):
        pass

    def test_sample(self):
        self.fail_1 += 1
        if self.fail_1 == 2:
            raise Exception()

    def test_shuffle(self):
        self.fail_2 += 1
        if self.fail_2 == 3:
            raise Exception()


def get_tests():
    test_funcs = ['test_choice', 'test_sample', 'test_shuffle']
    return [TestSequenceFunctions(func) for func in test_funcs]


if __name__ == '__main__':
    test_suite = unittest.TestSuite()

    repetitions = 3
    tests = get_tests()
    for __ in xrange(0, repetitions):
        test_suite.addTests(tests)

    CustomTextTestRunner(verbosity=2).run(test_suite)

Running the above code produces the following output:

>>> ================================ RESTART ================================
>>> 
[1/9] test_choice (__main__.TestSequenceFunctions) ... ok
[2/9] test_sample (__main__.TestSequenceFunctions) ... ok
[3/9] test_shuffle (__main__.TestSequenceFunctions) ... ok
[4/9] test_choice (__main__.TestSequenceFunctions) ... ok
[5/9] test_sample (__main__.TestSequenceFunctions) ... ERROR
[6/9] test_shuffle (__main__.TestSequenceFunctions) ... ok
[7/9] test_choice (__main__.TestSequenceFunctions) ... ok
[8/9] test_sample (__main__.TestSequenceFunctions) ... ok
[9/9] test_shuffle (__main__.TestSequenceFunctions) ... ERROR

======================================================================
ERROR: test_sample (__main__.TestSequenceFunctions)
----------------------------------------------------------------------
Test number: [5/9] 
Traceback (most recent call last):
  File "stackoverflow.py", line 75, in test_sample
    raise Exception()
Exception

======================================================================
ERROR: test_shuffle (__main__.TestSequenceFunctions)
----------------------------------------------------------------------
Test number: [9/9] 
Traceback (most recent call last):
  File "stackoverflow.py", line 80, in test_shuffle
    raise Exception()
Exception

----------------------------------------------------------------------
Ran 9 tests in 0.042s

FAILED (errors=2)
>>> 

Answer 2

You would have to subclass TextTestRunner, but I don't know how. I strongly suggest you re-examine your aversion to using nose. It's a very powerful tool that will easily solve your problem.