How to stop all tests from inside a test or setUp using unittest?

Question

I'm extending the python 2.7 unittest framework to do some function testing. One of the things I would like to do is to stop all the tests from running inside of a test, and inside of a setUpClass() method. Sometimes if a test fails, the program is so broken it is no longer of any use to keep testing, so I want to stop the tests from running.

I noticed that a TestResult has a shouldStop attribute, and a stop() method, but I'm not sure how to get access to that inside of a test.

Does anyone have any ideas? Is there a better way?

Answer 1

In case you are interested, here is a simple example how you could make a decision yourself about exiting a test suite cleanly with py.test:

# content of test_module.py import pytest counter = 0 def setup_function(func):     global counter     counter += 1     if counter >=3:         pytest.exit("decided to stop the test run")  def test_one():     pass def test_two():     pass def test_three():     pass 

and if you run this you get:

$ pytest test_module.py  ============== test session starts ================= platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1 test path 1: test_module.py  test_module.py ..  !!!! Exit: decided to stop the test run !!!!!!!!!!!! ============= 2 passed in 0.08 seconds ============= 

You can also put the py.test.exit() call inside a test or into a project-specific plugin.

Sidenote: py.test natively supports py.test --maxfail=NUM to implement stopping after NUM failures.

Sidenote2: py.test has only limited support for running tests in the traditional unittest.TestCase style.

Answer 2

Here's another answer I came up with after a while:

First, I added a new exception:

class StopTests(Exception): """ Raise this exception in a test to stop the test run.  """     pass 

then I added a new assert to my child test class:

def assertStopTestsIfFalse(self, statement, reason=''):     try:         assert statement                 except AssertionError:         result.addFailure(self, sys.exc_info()) 

and last I overrode the run function to include this right below the testMethod() call:

except StopTests:     result.addFailure(self, sys.exc_info())     result.stop() 

I like this better since any test now has the ability to stop all the tests, and there is no cpython-specific code.