Repeated single or multiple tests with Nose

Question

Similar to this question, I'd like to have Nose run a test (or all tests) n times -- but not in parallel.

I have a few hundred tests in a project; some are some simple unit tests. Others are integration tests w/ some degree of concurrency. Frequently when debugging tests I want to "hit" a test harder; a bash loop works, but makes for a lot of cluttered output -- no more nice single "." for each passing test. Having the ability to beat on the selected tests for some number of trials seems like a natural thing to ask Nose to do, but I haven't found it anywhere in the docs.

What's the simplest way to get Nose to do this (other than a bash loop)?

Answer 1

You can write a nose test as a generator, and nose will then run each function yielded:

def check_something(arg):
    # some test ...

def test_something():
    for arg in some_sequence:
        yield (check_something, arg)

Using nose-testconfig, you could make the number of test runs a command line argument:

from testconfig import config

# ...

def test_something():
    for n in range(int(config.get("runs", 1))):
        yield (check_something, arg)

Which you'd call from the command line with e.g.

$ nosetests --tc=runs:5

... for more than one run.

Alternatively (but also using nose-testconfig), you could write a decorator:

from functools import wraps
from testconfig import config

def multi(fn):
    @wraps(fn)
    def wrapper():
        for n in range(int(config.get("runs", 1))):
            fn()
    return wrapper

@multi
def test_something():
    # some test ...

And then, if you want to divide your tests into different groups, each with their own command line argument for the number of runs:

from functools import wraps
from testconfig import config

def multi(cmd_line_arg):
    def wrap(fn):
        @wraps(fn)
        def wrapper():
            for n in range(int(config.get(cmd_line_arg, 1))):
                fn()
        return wrapper
    return wrap

@multi("foo")
def test_something():
    # some test ...

@multi("bar")
def test_something_else():
    # some test ...

Which you can call like this:

$ nosetests --tc=foo:3 --tc=bar:7

Answer 2

You'll have to write a script to do this, but you can repeat the test names on the commandline X times.

nosetests testname testname testname testname testname testname testname

etc.