How do I run all Python unit tests in a directory?

Question

I have a directory that contains my Python unit tests. Each unit test module is of the form test_*.py. I am attempting to make a file called all_test.py that will, you guessed it, run all files in the aforementioned test form and return the result. I have tried two methods so far; both have failed. I will show the two methods, and I hope someone out there knows how to actually do this correctly.

For my first valiant attempt, I thought "If I just import all my testing modules in the file, and then call this unittest.main() doodad, it will work, right?" Well, turns out I was wrong.

import glob import unittest  testSuite = unittest.TestSuite() test_file_strings = glob.glob('test_*.py') module_strings = [str[0:len(str)-3] for str in test_file_strings]  if __name__ == "__main__":      unittest.main() 

This did not work, the result I got was:

$ python all_test.py   ---------------------------------------------------------------------- Ran 0 tests in 0.000s  OK 

For my second try, I though, ok, maybe I will try to do this whole testing thing in a more "manual" fashion. So I attempted to do that below:

import glob import unittest  testSuite = unittest.TestSuite() test_file_strings = glob.glob('test_*.py') module_strings = [str[0:len(str)-3] for str in test_file_strings] [__import__(str) for str in module_strings] suites = [unittest.TestLoader().loadTestsFromName(str) for str in module_strings] [testSuite.addTest(suite) for suite in suites] print testSuite   result = unittest.TestResult() testSuite.run(result) print result  #Ok, at this point I have a result #How do I display it as the normal unit test command line output? if __name__ == "__main__":     unittest.main() 

This also did not work, but it seems so close!

$ python all_test.py  <unittest.TestSuite tests=[<unittest.TestSuite tests=[<unittest.TestSuite tests=[<test_main.TestMain testMethod=test_respondes_to_get>]>]>]> <unittest.TestResult run=1 errors=0 failures=0>  ---------------------------------------------------------------------- Ran 0 tests in 0.000s  OK 

I seem to have a suite of some sort, and I can execute the result. I am a little concerned about the fact that it says I have only run=1, seems like that should be run=2, but it is progress. But how do I pass and display the result to main? Or how do I basically get it working so I can just run this file, and in doing so, run all the unit tests in this directory?

Answer 1

With Python 2.7 and higher you don't have to write new code or use third-party tools to do this; recursive test execution via the command line is built-in. Put an __init__.py in your test directory and:

python -m unittest discover <test_directory> # or python -m unittest discover -s <directory> -p '*_test.py' 

You can read more in the python 2.7 or python 3.x unittest documentation.

---

Update for 2021:

Lots of modern python projects use more advanced tools like pytest. For example, pull down matplotlib or scikit-learn and you will see they both use it.

It is important to know about these newer tools because when you have more than 7000 tests you need:

• more advanced ways to summarize what passes, skipped, warnings, errors
• easy ways to see how they failed
• percent complete as it is running
• total run time
• ways to generate a test report
• etc etc

Answer 2

In python 3, if you're using unittest.TestCase:

• You must have an empty (or otherwise) __init__.py file in your test directory (must be named test/)
• Your test files inside test/ match the pattern test_*.py. They can be inside a subdirectory under test/, and those subdirs can be named as anything.

Then, you can run all the tests with:

python -m unittest 

Done! A solution less than 100 lines. Hopefully another python beginner saves time by finding this.