Disable automatic test execution for some tests unless those are explicitly requested

Question

I have a few very slow tests and many short unittests. I would like to be able to run only the short unittests with plain nosetests command and if I decide it's time to run the slow tests, to be able to call them explicitly.

What I'd like to be able to do:

• run unittests but not the slow tests

  $ nosetests
  

  No special command is used - anyone who enters nosetests just out of curiosity would be satisfied in a few seconds.

• explicitly request the slow tests:

  $ nosetests --somehow-magicaly-invoke-slow-tests
  

  There is no ambiguity - I want the slow tests (or unittests and the slow test - it does not matter)

What have I tried:

I tried using nose.plugins.attrib:

from nose.plugins.attrib import attr
import time

@attr('slow')
def test_slow_one():
    time.sleep(5)

def test_unittest():
    assert True

but it in fact does almost the opposite of what I'm trying to accomplish - I have to specify extra commandline parameters to not run the slow tests. The commands are:

Run unittests but not the slow tests

$ nosetests -v -a '!slow'
test_a.test_unittest ... ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK

explicitly request the slow tests:

$ nosetests -v -a 'slow'
test_a.test_slow_one ... ok
----------------------------------------------------------------------
Ran 1 test in 5.005s
OK

And to make things worse, when someone runs just

$ nosetests -v
test_a.test_slow_one ... ok
test_a.test_unittest ... ok
----------------------------------------------------------------------
Ran 2 tests in 5.007s
OK

all tests including the slow ones will be run.

The question:

Is there a way to disable some tests so they won't get called with plain nosetests command, but can be run with some additional commandline parameters?

Current "solution":

I just moved all slow tests into separate files and named the files check_*.py instead of test_*.py so nosetests won't pick them up. When I want to run the slow tests I just specify the whole path to the check_*.py files, something like:

$ nosetests test/check_foo.py test/check_bar.py [...]

which is not very elegant.

Answer 1

You can easily use unittest.skipUnless() to that effect.

Skipping single tests

Simply decorate the methods of your test case that you want to conditionally skip with

@unittest.skipUnless(condition, reason)

For example, you could check for an environment variable SLOW_TESTS that you simply don't set in your automated CI environment, but set if and when you want to run your slow tests locally:

import os
import time
import unittest


SLOW_TESTS = os.environ.get('SLOW_TESTS', False)


    class TestFoo(unittest.TestCase):

        def test_upper(self):
            self.assertEqual('foo'.upper(), 'FOO')

        @unittest.skipUnless(SLOW_TESTS, "slow test")
        def test_something_slow(self):
            time.sleep(5)
            self.assertTrue(True)

Output for a regular run:

$ nosetests -v
test_something_slow (test_foo.TestFoo) ... SKIP: slow test
test_upper (test_foo.TestFoo) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.002s

OK (SKIP=1)

Output when setting the environment variable:

$ SLOW_TESTS=1 nosetests -v
test_something_slow (test_foo.TestFoo) ... ok
test_upper (test_foo.TestFoo) ... ok

----------------------------------------------------------------------
Ran 2 tests in 5.003s

OK

(Note that the Nose testrunner still says Ran 2 tests even though it skipped one. The skipped tests are indicated at the very end with (SKIP=n) and in the test results with SKIP or S in non-verbose mode).

Of course you could invert the behavior by using skipIf() using an environment variable like FAST_TESTS that you set in your CI setup.

Skipping an entire TestCase

If you want to skip all tests in a TestCase that you know to be slow or have some heavy setup, it may be more convenient to call TestCase.skipTest() explicitly (you can also do that from a single test if you need more fine grained control):

class TestFoo(unittest.TestCase):

    def setUp(self):
        if not SLOW_TESTS:
            self.skipTest("slow test")
        # some expensive setup

See Skipping tests and expected failures for more details on skipping tests.