Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multiple concurrent database transactions with Django?

Is it possible to use multiple concurrent transactions within one Django process?

Specifically, I've got two functions which should each execute concurrently (yielding between each other), but should each be in their own transaction. For example:

def process_things(callback, things):
    with start_transaction():
        for thing in things:
            obj = Thing.objects.create(…)
            callback(obj)

def callback(obj):
    with start_separate_transaction():
        …
        ThingProcessingLog.objects.create(…)

I'd like each function to be operating in an independent database transaction, so in this example, the ThingProcessingLog records will be created and visible immediately, but the Thing objects won't be visible until they have all been processed.

How can I do this with Django's ORM?

like image 255
David Wolever Avatar asked Mar 19 '14 21:03

David Wolever


1 Answers

I did a little test (but with no transaction but I think it might work)

I setup 2 dbs in the django's settings:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'mydb',
        'USER': 'mydb',
        'PASSWORD': 'mydb',
        'HOST': '127.0.0.1',
        'PORT': '3306',
    },
    'alias': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'mydb',
        'USER': 'mydb',
        'PASSWORD': 'mydb',
        'HOST': '127.0.0.1',
        'PORT': '3306',
    }
}

Then in a django shell I did that:

Table1.objects.using('default').all()
Table1.objects.using('alias').all()

Table2.objects.using('default').all()
Table2.objects.using('alias').all()

In the same time I did this in mysql

mysql> show processlist;
+------+------+-----------------+------+---------+------+-------+------------------+
| Id   | User |     Host        | db   | Command | Time | State | Info             |
+------+------+-----------------+------+---------+------+-------+------------------+
| 1314 | mydb | localhost:40224 | mydb | Query   |    0 | NULL  | show processlist |
| 1315 | mydb | localhost:40225 | mydb | Sleep   |    5 |       | NULL             |
| 1316 | mydb | localhost:40226 | mydb | Sleep   |    5 |       | NULL             |
+------+------+-----------------+------+---------+------+-------+------------------+

So I suppose you will have a transaction for each db (which are actually the same db).

like image 165
MoiTux Avatar answered Sep 22 '22 13:09

MoiTux