Run specific Django tests (with django-nose?)

Question

I am having a very complicated tests.py file.

Actually the tests classes and methods are generated at run time w/ type (to account for data listed in auxiliary files). I am doing things in the following fashion (see below for more code):

klass = type(name, (TestCase,), attrs)
setattr(current_module, name, klass)

FYI, with the usual django test runner, all those tests get run when doing ./manage.py test myapp (thanks to the setattr shown above).

What I want to do is run only part of those tests, without listing their names by hand.

For example, I could give each test "tags" in the class names or method names so that I could filter on them. For example I would then perform: run all tests which method name contains the string "test_postgres_backend_"

I tried using django-nose because of nose's -m option, which should be able to select tests based on regular expressions, an ideal solution to my problem.

Unfortunately, here is what is happening when using django-nose as the django test runner:

• ./manage.py test myapp is not finding automatically the type-generated test classes (contrarily to the django test runner)
• neither ./manage.py test -m ".*" myapp nor ./manage.py test myapp -m ".*" find ANY test, even if normal TestCase classes are present in the file

So:

• Do you have another kind of solution to my general problem, rather than trying to use django-nose -m?
• With django-nose, do you know how to make the -m work?

---

mcve

Add the following to an empty myapp/tests.py file:

from django.test import TestCase
from sys import modules


current_module = modules[__name__]


def passer(self, *args, **kw):
    self.assertEqual(1, 1)


def failer(self, *args, **kw):
    self.assertEqual(1, 2)


# Create a hundred ...
for i in xrange(100):
    # ... of a stupid TestCase class that has 1 method that passes if `i` is
    # even and fails if `i` is odd
    klass_name = "Test_%s" % i
    if i % 2:  # Test passes if even
        klass_attrs = {
            'test_something_%s' % i: passer
        }
    else:      # Fail if odd
        klass_attrs = {
            'test_something_%s' % i: failer
        }
    klass = type(klass_name, (TestCase,), klass_attrs)

    # Set the class as "child" of the current module so that django test runner
    # finds it
    setattr(current_module, klass_name, klass)

If makes for this output run (in alphab order) by django test runnner:
F.F.F.F.F.F.FF.F.F.F.F..F.F.F.F.F.FF.F.F.F.F..F.F.F.F.F.FF.F.F.F.F..F.F.F.F.F.FF.F.F.F.F..F.F.F.F.F..

If you change to django_nose test runner, nothing happens on ./manage.py test myapp.

After fixing this, I would then like would be able to run only the test methods which name end with a 0 (or some other kind of regexable filtering)

Answer 1

The problem you ran into is that Nose determines whether or not to include a method into the set of tests to run by looking at the name recorded on the function itself, rather than the attribute that gives access to the function. If I rename your passer and failer to test_pass and test_fail then Nose is able to find the tests. So the functions themselves have to be named in a way that will be matched by what is given to -m (or its default value).

Here's the modified code that gives the expected results:

from django.test import TestCase
from sys import modules

current_module = modules[__name__]

def test_pass(self, *args, **kw):
    self.assertEqual(1, 1)

def test_fail(self, *args, **kw):
    self.assertEqual(1, 2)

# Create a hundred ...
for i in xrange(100):
    # ... of a stupid TestCase class that has 1 method that passes if `i` is
    # even and fails if `i` is odd
    klass_name = "Test_%s" % i
    if i % 2:  # Test passes if even
        klass_attrs = {
            'test_something_%s' % i: test_pass
        }
    else:      # Fail if odd
        klass_attrs = {
            'test_something_%s' % i: test_fail
        }
    klass = type(klass_name, (TestCase,), klass_attrs)

    # Set the class as "child" of the current module so that django test runner
    # finds it
    setattr(current_module, klass_name, klass)

# This prevents Nose from seeing them as tests after the loop is over.
test_pass = None
test_fail = None

Without the final two assignments to None, Nose will consider the two top level functions to be module-level tests and will run them in addition to the tests in the classes.

Another way to get the same results would be to define __test__ on your two functions:

def passer(self, *args, **kw):
    self.assertEqual(1, 1)
passer.__test__ = 1

def failer(self, *args, **kw):
    self.assertEqual(1, 2)
failer.__test__ = 1

And at the end of the file:

# This prevents Nose from seeing them as tests after the loop is over.
passer = None
failer = None

Nose looks for the presence of these on functions and if present and set to a value is considered to be true, it will take the function as a test case.

The logic governing the selection of methods can be found in Nose's selector.py file, at the wantMethod method:

def wantMethod(self, method):
        """Is the method a test method?
        """
        try:
            method_name = method.__name__
        except AttributeError:
            # not a method
            return False
        if method_name.startswith('_'):
            # never collect 'private' methods
            return False
        declared = getattr(method, '__test__', None)
        if declared is not None:
            wanted = declared
        else:
            wanted = self.matches(method_name)
        plug_wants = self.plugins.wantMethod(method)
        if plug_wants is not None:
            wanted = plug_wants
        log.debug("wantMethod %s? %s", method, wanted)
        return wanted

I'm not seeing a clear way to use -m to run only some tests the way you want it. The problem is that -m matches file, directorie, module, class, and function names equally. If you set something like -m0$ then all the individual parts I just listed must match the regular expression for the test to be selected. (Nose does not combine them and then match on the combination.) It is possible to list tests individually on the command line but this is a poor substitute to a regular expression match.