Skip unittest test without decorator syntax

Question

I have a suite of tests that I have loaded using TestLoader's (from the unittest module) loadTestsFromModule() method, i.e.,

suite = loader.loadTestsFromModule(module)

This gives me a perfectly ample list of tests that works fine. My problem is that the test harness I'm working with sometimes needs to skip certain tests based on various criteria. What I want to do is something like this:

for test in suite:
    mark the test as 'to-skip' if it meets certain criteria

Note that I can't just remove the test from the list of tests because I want the unittest test runner to actually skip the tests, add them to the skipped count, and all of that jazz.

The unittest documentation suggests using decorators around the test methods or classes. Since I'm loading these tests from a module and determining to skip based on criteria not contained within the tests themselves, I can't really use decorators. Is there a way I can iterate over each individual test and some how mark it as a "to-skip" test without having to directly access the test class or methods within the class?

Answer 1

Using unittest.TestCase.skipTest:

import unittest

class TestFoo(unittest.TestCase):
    def setUp(self): print('setup')
    def tearDown(self): print('teardown')
    def test_spam(self): pass
    def test_egg(self): pass
    def test_ham(self): pass

if __name__ == '__main__':
    import sys
    loader = unittest.loader.defaultTestLoader
    runner = unittest.TextTestRunner(verbosity=2)
    suite = loader.loadTestsFromModule(sys.modules['__main__'])
    for ts in suite:
        for t in ts:
            if t.id().endswith('am'): # To skip `test_spam` and `test_ham`
                setattr(t, 'setUp', lambda: t.skipTest('criteria'))
    runner.run(suite)

prints

test_egg (__main__.TestFoo) ... setup
teardown
ok
test_ham (__main__.TestFoo) ... skipped 'criteria'
test_spam (__main__.TestFoo) ... skipped 'criteria'

----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK (skipped=2)


----------------------------------------------------------------------
Ran 3 tests in 0.002s

OK (skipped=2)

UPDATE

Updated the code to patch setUp instead of test method. Otherwise, setUp/tearDown methods will be executed for test to be skipped.

NOTE

unittest.TestCase.skipTest (Test skipping) was introduced in Python 2.7, 3.1. So this method only work in Python 2.7+, 3.1+.

Answer 2

This is a bit of a hack, but because you only need to raise unittest.SkipTest you can walk through your suite and modify each test to raise it for you instead of running the actual test code:

import unittest
from unittest import SkipTest

class MyTestCase(unittest.TestCase):
    def test_this_should_skip(self):
        pass

    def test_this_should_get_skipped_too(self):
        pass

def _skip_test(reason):
    raise SkipTest(reason)

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(MyTestCase)
    for test in suite:
        skipped_test_method = lambda: _skip_test("reason")
        setattr(test, test._testMethodName, skipped_test_method)
    unittest.TextTestRunner(verbosity=2).run(suite)

When I run this, this is the output I get:

test_this_should_get_skipped_too (__main__.MyTestCase) ... skipped 'reason'
test_this_should_skip (__main__.MyTestCase) ... skipped 'reason'

----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK (skipped=2)