Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Automatic db router for migrations in django

I am building a Django powered site and I want to have separate databases for some of the apps, I build a flexible router that routes each app to predefined database, this works fine. The problem is that when I am migrating my models I have to set the --database parameter every time and I find this annoying and redundant. Also many times I flooded my default database with tables from migrated app (by forgetting to add --database). I experimented with allow_migrate(...) function in my router but all I could achieve is safety mechanism that will not run the migration if I forget to specify the database. My question is: is there a way to set up a automatic database selection for models migrations in Django? My approach might be wrong, it surprises me that no one seems to have done that before.

like image 314
Raff89 Avatar asked Aug 24 '16 21:08

Raff89


2 Answers

For anyone facing the issue I was having, here is what I finally settled on:

routers.py:

from django.conf import settings
class DatabaseAppsRouter(object):

    def db_for_read(self, model, **hints):
        #  your db routing

    def db_for_write(self, model, **hints):
        #  your db routing

    def allow_relation(self, obj1, obj2, **hints):
        #  your db routing

    def allow_syncdb(self, db, model):
        #  your db routing

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if db in settings.DATABASE_APPS_MAPPING.values():
            return settings.DATABASE_APPS_MAPPING.get(app_label) == db
        elif app_label in settings.DATABASE_APPS_MAPPING:
            return False

settings.py

DATABASE_ROUTERS = ['project.routers.DatabaseAppsRouter']
DATABASE_APPS_MAPPING = {'contenttypes': 'default',
                     'auth': 'default',
                     'admin': 'default',
                     'sessions': 'default',
                     'messages': 'default',
                     'staticfiles': 'default',
                     'myapp': 'mydb',
                     }

this setup lets me to run two commands:

python manage.py migrate
python manage.py migrate --database=mydb

which is still one to many but at least now I cannot mistakenly migrate myapp to the default database which happened more times that I want to admit.


Warning - rant:

Up to this point I was finding the Django design to be quircky but very logical and programmer friendly, like python itself. But the way that migrations interact with multiple database setups is really inconvenient makes it so easy to flood your default database with unwanted tables. Discussion linked by kevin in his answer shows that I am not the only one to have this kind of difficulties.

like image 180
Raff89 Avatar answered Sep 22 '22 19:09

Raff89


I don't know of any way to do that automatically. However, a simple approach might be to write your own version of the migrate command that calls the Django migrate command multiple times with the appropriate --database and app arguments.

According to the author of Django migrations, this is a conscious design decision, so I wouldn't expect it to change: "Just like syncdb, migrate only runs on one database at a time, so you must execute it individually for each database, as you suggest. This is by design."

like image 27
Kevin Christopher Henry Avatar answered Sep 24 '22 19:09

Kevin Christopher Henry