Making Nose fail slow tests

Question

I want to my tests to fail if they take longer than a certain time to run (say 500ms) because it sucks when a load of slightly slow tests mount up and suddenly you have this big delay every time you run the test suite. Are there any plugins or anything for Nose that do this already?

Answer 1

For cases where where timing is important (e.g. realtime requirements):

http://nose.readthedocs.org/en/latest/testing_tools.html

nose.tools.timed(limit)

Test must finish within specified time limit to pass.

Example use:

 from nose.tools import timed
 @timed(.1)
 def test_that_fails():
     time.sleep(.2)

Answer 2

I respectfully suggest that changing the meaning of "broken" is a bad idea.

The meaning of a failed/"red" test should never be anything other than "this functionality is broken". To do anything else risks diluting the value of the tests.

If you implement this and then next week a handful of tests fail, would it be an indicator that

• Your tests are running slowly?
• The code is broken?
• Both of the above at the same time?

I suggest it would be better to gather MI from your build process and monitor it in order to spot slow tests building up, but let red mean "broken functionality" rather then "broken functionality and/or slow test."