Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting data from multiple databases with same tablenames in django

I need to get data from different imported MySQL-databases in Django (Django 1.11.7, Python 3.5.2). I run manage.py inspectdb --database '<db>' and then use the models in django.

Until now, I only had to access tables with different names. For this purpose, I used the using keyword in the queryset to specify the appropriate database and then concatenated the result, like this:

from ..models.db1 import Members
from ..models.db2 import Actor

context['db1_data'] = Members.objects.using('db1').filter...
context['db2_data'] = Actor.objects.using('db1').filter...

context["member_list"] = list(chain(
    context["db1_data"],
    context["db2_data"],
))

return context

Now I have the problem that there are tables with the same model names in two databases. I get the following error when using the above-mentioned method (I substituted the names):

RuntimeError: Conflicting '<table-name>' models in application '<app>': <class '<app>.<subfolder>.models.<db1>.<table-name>'> and <class '<app>.<subfolder>.models.<db2>.<table-name>'>.

I already tried importing the model with a different name, like this:

from ..models.db3 import Members as OtherMembers

but the error still comes up.

Shouldn't from ..models.db1 and from ..models.db2 be clear enough for Django to spot the difference between the two models?

One option would probably be to rename the models themselves, but that would mean to rename every database model with the same names. Since I will use many more databases in the future, this is not an option for me.

I tried from models import db1, db2 and then db1.Members etc., which still raises the error.

I read about the meta db_table = 'dbname.tablename'-option, but since the model is auto-generated through inspectdb, it already has something like this on every class:

class MyModel(models.Model):
    <models>

    class Meta:
        managed = False
        db_table = 'my_model'

As described before, the other database has the exact same model and hence the same Meta classes. I can't and don't want to change every Meta class.


EDIT:

My project structure looks like this:

app
    -> admin.py
    -> ...
    -> models.py
    -> views.py
    subfolder
        -> models
            -> db1.py
            -> db2.py
        -> views
            -> db1.py
            -> db2.py
like image 275
sqe Avatar asked Dec 11 '17 10:12

sqe


2 Answers

Assuming that you have set up your multiple databases correctly:

  1. Have you tried to add a Custom Router?
    If not follow the example given on the documentation link.

  2. Have you tried to use a Custom Manager for your models?

    Create a manager for each model, like this:

    class YourModelManagerX(models.Manager):
        def get_queryset(self, *args, **kwargs):
            return super().get_queryset(*args, **kwargs).using('your_db_X')
    

    And then add it to your appropriate model as the objects field:

    class YourModel(models.Model):
        ...
        fields
        ...
        objects = YourManagerX()
    
        class Meta:
            managed = False
    

You may need to try both at once.

like image 114
John Moutafis Avatar answered Oct 22 '22 05:10

John Moutafis


If db1.Members and db3.Members have the same definition, you do not have to redeclare the Members class separately for each database.

Models.py

...
class Members(models.Model): # Only declared once ever!
    ....

then,

from Models import Members

context['db1_data'] = Members.objects.using('db1').filter...
context['db3_data'] = Members.objects.using('db3').filter...

... # continue processing

Shouldn't from ..models.db1 and from ..models.db2 be clear enough for django to spot the difference between the two models?

Django Models are not database-specific, more like schema-specific, so if you have the same table in two different databases, one class extending model.Models suffices. Then, when you attempt to retrieve the objects, either specify the database with using(), or using routers, which you can read about in the Django docs https://docs.djangoproject.com/en/2.0/topics/db/multi-db/#an-example

like image 1
codelessbugging Avatar answered Oct 22 '22 05:10

codelessbugging