django-discover-runner and XML reports?

Question

I have a working Django app which has been successfully using unittest-xml-reporting to produce XML reports from my unittests.

However, the project is growing rapidly and I want to break up my tests into separate files within each app. I therefore installed django-discover-runner which finds all my test files and runs them successfully.

However, django-discover-runner does not produce XML reports, which I need (for Bamboo).

I've found this:

http://www.stevetrefethen.com/blog/Publishing-Python-unit-test-results-in-Jenkins.aspx

and have tried to implement that suggestion (in each of my test.py files), but no XML is produced.

How can I use both django-discover-runner and unittest-xml-reporting to discover my tests and produce XML reports?

Answer 1

Since this question was asked, the unittest-xml-reporting project has added support for the new Django DiscoverRunner class. You can just set the test runner in your Django settings file:

TEST_RUNNER = 'xmlrunner.extra.djangotestrunner.XMLTestRunner'

It will run the same tests as the DiscoverRunner would.

Answer 2

So it turns out that the solution to this is much easier than I expected. For any other n00bs who may be looking to do this:

The short answer is that I simply cobbled together the two test runners provided by django-discover-runner and unittest-xml-reporting into a custom test runner:

from django.conf import settings
from django.test.utils import setup_test_environment, teardown_test_environment
import xmlrunner
from django.core.exceptions import ImproperlyConfigured
from django.test import TestCase
from django.test.simple import DjangoTestSuiteRunner, reorder_suite
from django.utils.importlib import import_module

try:
    from django.utils.unittest import defaultTestLoader
except ImportError:
    try:
        from unittest2 import defaultTestLoader  # noqa
    except ImportError:
        raise ImproperlyConfigured("Couldn't import unittest2 default "
                               "test loader. Please use Django >= 1.3 "
                               "or go install the unittest2 library.")

### CUSTOM RUNNER NAME
class myTestRunner(DjangoTestSuiteRunner):
    ### THIS SECTION FROM UNITTESTS-XML-REPORTING
    def build_suite(self, test_labels, extra_tests=None, **kwargs):
        suite = None
        root = getattr(settings, 'TEST_DISCOVER_ROOT', '.')
        top_level = getattr(settings, 'TEST_DISCOVER_TOP_LEVEL', None)
        pattern = getattr(settings, 'TEST_DISCOVER_PATTERN', 'test*.py')

        if test_labels:
            suite = defaultTestLoader.loadTestsFromNames(test_labels)
            # if single named module has no tests, do discovery within it
            if not suite.countTestCases() and len(test_labels) == 1:
                suite = None
                root = import_module(test_labels[0]).__path__[0]

        if suite is None:
            suite = defaultTestLoader.discover(root,
                pattern=pattern, top_level_dir=top_level)

        if extra_tests:
            for test in extra_tests:
                suite.addTest(test)

        return reorder_suite(suite, (TestCase,))

    ###THIS SECTION FROM DJANGO-DISCOVER-RUNNER
    def run_tests(self, test_labels, extra_tests=None, **kwargs):
        """
        Run the unit tests for all the test labels in the provided list.
        Labels must be of the form:
         - app.TestClass.test_method
        Run a single specific test method
         - app.TestClass
        Run all the test methods in a given class
         - app
        Search for doctests and unittests in the named application.

        When looking for tests, the test runner will look in the models and
        tests modules for the application.

        A list of 'extra' tests may also be provided; these tests
        will be added to the test suite.

        Returns the number of tests that failed.
        """
        setup_test_environment()

        settings.DEBUG = False

        verbosity = getattr(settings, 'TEST_OUTPUT_VERBOSE', 1)
        if isinstance(verbosity, bool):
            verbosity = (1, 2)[verbosity]
        descriptions = getattr(settings, 'TEST_OUTPUT_DESCRIPTIONS', False)
        output = getattr(settings, 'TEST_OUTPUT_DIR', '.')

        suite = self.build_suite(test_labels, extra_tests)

        old_config = self.setup_databases()

        result = xmlrunner.XMLTestRunner(
            verbosity=verbosity, descriptions=descriptions,
            output=output).run(suite)

        self.teardown_databases(old_config)
        teardown_test_environment()

        return len(result.failures) + len(result.errors)

This should be saved somewhere suitable within your project. In your test settings file (test_settings.py - as per django-discover-runner instructions), set the Test Runner:

TEST_RUNNER = '<your-django-project>.customTestRunner.myTestRunner'

You would then use (again, as per django-discover-runner instructions):

django-admin.py test --settings=myapp.test_settings

This solution allows me to use django-discover-runner's features to discover all the test files across my project - specified by django-discover-runner's TEST_DISCOVER_PATTERN option - and still output XML reports as required by Bamboo. Big thanks to the authors of the original code:

django-discover-runner

unittest-xml-reports