Django ignores router when running tests?

Question

I have a django application that uses 2 database connections:

1. To connect to the actual data the app is to produce
2. To a reference master data system, that is maintained completely outside my control

The issue that I'm having, is that my webapp can absolutely NOT touch the data in the 2nd database. I solved most of the issues by using 2 (sub)apps, one for every database connection. I created a router file that router any migration, and writing to the first app

I also made all the models in the 2nd app non managed, using the

model.meta.managed = False

option.

To be sure, the user I connect to the 2nd database has read only access

This works fine for migrations and running. However, when I try to run tests using django testcase, Django tries to delete and create a test_ database on the 2nd database connection.

How can I make sure that Django will NEVER update/delete/insert/drop/truncate over the 2nd connection

How can I run tests, that do not try to create the second database, but do create the first.

Thanks!

edited: code

model (for the 2nd app, that should not be managed):

from django.db import models


class MdmMeta(object):
    db_tablespace = 'MDM_ADM'
    managed = False
    ordering = ['name']


class ActiveManager(models.Manager):
    def get_queryset(self):
        return super(ActiveManager, self).get_queryset().filter(lifecyclestatus='active')


class MdmType(models.Model):
    entity_guid = models.PositiveIntegerField(db_column='ENTITYGUID')
    entity_name = models.CharField(max_length=255, db_column='ENTITYNAME')

    entry_guid = models.PositiveIntegerField(primary_key=True, db_column='ENTRYGUID')

    name = models.CharField(max_length=255, db_column='NAME')
    description = models.CharField(max_length=512, db_column='DESCRIPTION')

    lifecyclestatus = models.CharField(max_length=255, db_column='LIFECYCLESTATUS')

    # active_manager = ActiveManager()

    def save(self, *args, **kwargs):
        raise Exception('Do not save MDM models!')

    def delete(self, *args, **kwargs):
        raise Exception('Do not delete MDM models!')

    def __str__(self):
        return self.name

    class Meta(MdmMeta):
        abstract = True


# Create your models here.
class MdmSpecies(MdmType):
    class Meta(MdmMeta):
        db_table = 'MDM_SPECIES'
        verbose_name = 'Species'
        verbose_name_plural = 'Species'


class MdmVariety(MdmType):
    class Meta(MdmMeta):
        db_table = 'MDM_VARIETY'
        verbose_name = 'Variety'
        verbose_name_plural = 'Varieties'

...

router:

__author__ = 'CoesseWa'

class MdmRouter(object):

    def db_for_read(self, model, **hints):
        if model._meta.app_label == 'mdm':
            # return 'default'
            return 'mdm_db'   # trying to use one database connection
        return 'default'

    def db_for_write(self, model, **hints):
        return 'default'

    def allow_relation(self, obj1, obj2, **hints):
        return None

    def allow_migrate(self, db, model):
        if model._meta.app_label == 'mdm':
            return False

settings:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.oracle',
        'NAME': '(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=%s)(PORT=1521)))(CONNECT_DATA=(SID=%s)))'
                % (get_env_variable('LIMS_MIGRATION_HOST'), get_env_variable('LIMS_MIGRATION_SID')),
        'USER': 'LIMS_MIGRATION',
        'PASSWORD': get_env_variable('LIMS_MIGRATION_PASSWORD'),
    },
    'mdm_db': {
        'ENGINE': 'django.db.backends.oracle',
        'NAME': '(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=GB3P)(PORT=1521)))'
                '(CONNECT_DATA=(SID=GB3P)))',
        'USER': 'MDM',
        'PASSWORD': get_env_variable('MDM_DB_PASSWORD'),
    },
}

one testcase:

from django.test.testcases import TestCase

__author__ = 'CoesseWa'


class ModelTest(TestCase):

    def test_getting_guid_for_mdm_field(self):
        self.assertIsNotNone(1)

output from when running this tests:

... 
Destroying old test user...

(before this point, django creates the test database for my first connection = OK)

Creating test user...

=> This next lines should never happen. Fails because I use a read only user (luckily)

Creating test database for alias 'mdm_db'...

Failed (ORA-01031: insufficient privileges 
Got an error creating the test database: ORA-01031: insufficient privileges

Answer 1

I solved this by changing the DATABASES.TEST definition. I added the TEST['MIRROR'] = 'default' to the mdm_db database entry.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.oracle',
        'NAME': '(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=%s)(PORT=1521)))(CONNECT_DATA=(SID=%s)))'
                % (get_env_variable('LIMS_MIGRATION_HOST'), get_env_variable('LIMS_MIGRATION_SID')),
        'USER': 'LIMS_MIGRATION',
        'PASSWORD': get_env_variable('LIMS_MIGRATION_PASSWORD'),
    },
    'mdm_db': {
        'ENGINE': 'django.db.backends.oracle',
        'NAME': '(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=GB3P)(PORT=1521)))'
                '(CONNECT_DATA=(SID=GB3P)))',
        'USER': 'MDM',
        'PASSWORD': get_env_variable('MDM_DB_PASSWORD'),
        'TEST': {
            'MIRROR': 'default',  # Added this setting
        }
    },
}

According to the documentation this option can be abused to skip database creation:

However, the replica database has been configured as a test mirror (using the MIRROR test setting), indicating that under testing, replica should be treated as a mirror of default.

When the test environment is configured, a test version of replica will not be created. Instead the connection to replica will be redirected to point at default.

Running my tests now skips creation of the second database.

Thanks for all the input!!

Answer 2

Use the --settings flag with the test command. In the path.to.test.py module, a la python manage.py test --settings=app.settings.test. There is no need to muck around with routes, just make sure that you invoke the tests with the settings flag whenever and where ever you call it.

In app.settings.test.py, redefine your DATABASES datastructure:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': '',
        'USER': '',
        'PASSWORD': '',
    },
    'mdm_db': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': '',
        'USER': '',
        'PASSWORD': '',
    },
}

This allows you to use a separate database when running tests. Additionally, if you use sqlite3 as your engine, you'll find that tests run very fast as the database is in memory.

Using sqlite3 databases for testing means that even hundreds of tests can be run within seconds. As a result, you can run your tests very frequently. I typically map a key to save my work and run my tests with one action:

map ,t :up\|!python manage.py test --settings=app.settings.test

Hope that is is helpful!