unittest.py doesn't play well with trace.py - why?

Question

Wow. I found out tonight that Python unit tests written using the unittest module don't play well with coverage analysis under the trace module. Here's the simplest possible unit test, in foobar.py:

import unittest

class Tester(unittest.TestCase):
    def test_true(self):
        self.assertTrue(True)

if __name__ == "__main__":
    unittest.main()

If I run this with python foobar.py, I get this output:

 .
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Great. Now I want to perform coverage testing as well, so I run it again with python -m trace --count -C . foobar.py, but now I get this:

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

No, Python, it's not OK - you didn't run my test! It seems as though running in the context of trace somehow gums up unittest's test detection mechanism. Here's the (insane) solution I came up with:

import unittest

class Tester(unittest.TestCase):
    def test_true(self):
        self.assertTrue(True)

class Insane(object):
    pass

if __name__ == "__main__":
    module = Insane()
    for k, v in locals().items():
        setattr(module, k, v)

    unittest.main(module)

This is basically a workaround that reifies the abstract, unnameable name of the top-level module by faking up a copy of it. I can then pass that name to unittest.main() so as to sidestep whatever effect trace has on it. No need to show you the output; it looks just like the successful example above.

So, I have two questions:

1. What is going on here? Why does trace screw things up for unittest?

2. Is there an easier and/or less insane way to get around this problem?

Answer 1

A simpler workaround is to pass the name of the module explicitly to unittest.main:

import unittest

class Tester(unittest.TestCase):
    def test_true(self):
        self.assertTrue(True)

if __name__ == "__main__":
    unittest.main(module='foobar')

trace messes up test discovery in unittest because of how trace loads the module it is running. trace reads the module source code, compiles it, and executes it in a context with a __name__ global set to '__main__'. This is enough to make most modules behave as if they were called as the main module, but doesn't actually change the module which is registered as __main__ in the Python interpreter. When unittest asks for the __main__ module to scan for test cases, it actually gets the trace module called from the command line, which of course doesn't contain the unit tests.

coverage.py takes a different approach of actually replacing which module is called __main__ in sys.modules.

Answer 2

I don't know why trace doesn't work properly, but coverage.py does:

$ coverage run foobar.py
.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK
$ coverage report
Name     Stmts   Miss  Cover
----------------------------
foobar       6      0   100%