Python run unittests continuously or each test multiple times

Question

I have wrote unit test cases to test my application. and it is working as expected with out issues.

below is some sample testcase

import os
import unittest

class CreateUser(unittest.TestCase):
    def setUp(self):
        pass

    def tearDown(self):
        pass
    def test_send_message(self):
        #my script goes here
        print "success"
if __name__ == '__main__':
    unittest.main()

If i run this test it executing as expected but I want to run this test case 'N' no of times,

for that i added for loop in main function, also its running only one time, code i used as below

 if __name__ == '__main__':
    for i in range(1, 5):
         unittest.main()  

I also used schedule lib to run test every 10 mins but no luck

Is there any way to run this test case multiple times or any other work around that i am missing or any other continuous build tool to achive this?

Thanks in advance

Answer 1

First, a little bit of caution.

Why do you want to run the same test five times? I don't want to make assumptions without seeing your code, but this is a pretty serious code smell. A unit test must be repeatable, and if running it five times in a row does not give the same result as running it once, then something in the test is not repeatable. In particular, if early runs of the test are creating side effects used by later runs, or if there is some kind of random number involved, both of those are very bad situations that need to be repaired rather than running the test multiple times. Given just the information that we have here, it seems very likely that the best advice will be don't run that test multiple times!

Having said that, you have a few different options.

1. Put the loop inside the test

Assuming there is something meaningful about calling a function five times, it is perfectly reasonable to do something like:

def test_function_calls(self):
    for _ in xrange(1, 5):
        self.assertTrue(f())

2. Since you mentioned the nose tag, you have some options for parameterized tests, which usually consists of running the same (code) test over different input values. If you used something like https://github.com/wolever/nose-parameterized , then your result might be something like this:

@parameterized.expand([('atest', 'a', 1), ('btest', 'b', 2)])
def test_function_calls(self, name, input, expected):
    self.assertEqual(expected, f(input))

Parameterized tests are, as the name implies, typically for checking one code test with several pieces of data. You can have a list with dummy data if you just want the test to run several times, but that's another suspicious code structure that goes back to my original point.

Bonus side note: almost all "continuous" build tools are set up to trigger builds/tests/etc. on specific conditions or events, like when code is submitted to a repository. It is very unusual for them to simply run tests continuously.

I've done my best to answer your question here, but I feel like something is missing. You may want to clarify exactly what you are trying to accomplish in order to get the best answer.