Recursive unittest discover

Question

I have a package with a directory "tests" in which I'm storing my unit tests. My package looks like:

. ├── LICENSE ├── models │   └── __init__.py ├── README.md ├── requirements.txt ├── tc.py ├── tests │   ├── db │   │   └── test_employee.py │   └── test_tc.py └── todo.txt 

From my package directory, I want to be able to find both tests/test_tc.py and tests/db/test_employee.py. I'd prefer not to have to install a third-party library (nose or etc) or have to manually build a TestSuite to run this in.

Surely there's a way to tell unittest discover not to stop looking once it's found a test? python -m unittest discover -s tests will find tests/test_tc.py and python -m unittest discover -s tests/db will find tests/db/test_employee.py. Isn't there a way to find both?

Answer 1

In doing a bit of digging, it seems that as long as deeper modules remain importable, they'll be discovered via python -m unittest discover. The solution, then, was simply to add a __init__.py file to each directory to make them packages.

. ├── LICENSE ├── models │   └── __init__.py ├── README.md ├── requirements.txt ├── tc.py ├── tests │   ├── db │   │   ├── __init__.py       # NEW │   │   └── test_employee.py │   ├── __init__.py           # NEW │   └── test_tc.py └── todo.txt 

So long as each directory has an __init__.py, python -m unittest discover can import the relevant test_* module.

Answer 2

If you're okay with adding a __init__.py file inside tests, you can put a load_tests function there that will handle discovery for you.

If a test package name (directory with __init__.py) matches the pattern then the package will be checked for a 'load_tests' function. If this exists then it will be called with loader, tests, pattern.

If load_tests exists then discovery does not recurse into the package, load_tests is responsible for loading all tests in the package.

I'm far from confident that this is the best way, but one way to write that function would be:

import os import pkgutil import inspect import unittest  # Add *all* subdirectories to this module's path __path__ = [x[0] for x in os.walk(os.path.dirname(__file__))]  def load_tests(loader, suite, pattern):     for imp, modname, _ in pkgutil.walk_packages(__path__):         mod = imp.find_module(modname).load_module(modname)         for memname, memobj in inspect.getmembers(mod):             if inspect.isclass(memobj):                 if issubclass(memobj, unittest.TestCase):                     print("Found TestCase: {}".format(memobj))                     for test in loader.loadTestsFromTestCase(memobj):                         print("  Found Test: {}".format(test))                         suite.addTest(test)      print("=" * 70)     return suite 

Pretty ugly, I agree.

First you add all subdirectories to the test packages's path (Docs).

Then, you use pkgutil to walk the path, looking for packages or modules.

When it finds one, it then checks the module members to see whether they're classes, and if they're classes, whether they're subclasses of unittest.TestCase. If they are, the tests inside the classes are loaded into the test suite.

So now, from inside your project root, you can type

python -m unittest discover -p tests 

Using the -p pattern switch. If all goes well, you'll see what I saw, which is something like:

Found TestCase: <class 'test_tc.TestCase'>   Found Test: testBar (test_tc.TestCase)   Found Test: testFoo (test_tc.TestCase) Found TestCase: <class 'test_employee.TestCase'>   Found Test: testBar (test_employee.TestCase)   Found Test: testFoo (test_employee.TestCase) ====================================================================== .... ---------------------------------------------------------------------- Ran 4 tests in 0.001s  OK 

Which is what was expected, each of my two example files contained two tests, testFoo and testBar each.

Edit: After some more digging, it looks like you could specify this function as:

def load_tests(loader, suite, pattern):     for imp, modname, _ in pkgutil.walk_packages(__path__):         mod = imp.find_module(modname).load_module(modname)         for test in loader.loadTestsFromModule(mod):             print("Found Tests: {}".format(test._tests))             suite.addTests(test) 

This uses the loader.loadTestsFromModule() method instead of the loader.loadTestsFromTestCase() method I used above. It still modifies the tests package path and walks it looking for modules, which I think is the key here.

The output looks a bit different now, since we're adding a found testsuite at a time to our main testsuite suite:

python -m unittest discover -p tests Found Tests: [<test_tc.TestCase testMethod=testBar>, <test_tc.TestCase testMethod=testFoo>] Found Tests: [<test_employee.TestCase testMethod=testBar>, <test_employee.TestCase testMethod=testFoo>] ====================================================================== .... ---------------------------------------------------------------------- Ran 4 tests in 0.000s  OK 

But we still get the 4 tests we expected, in both classes, in both subdirectories.