Does django cache model data between queries?

Question

I am using django 1.6, with mysql 5.6 as the database with innodb tables. Debug is set to false in my settings file.

Within a script, I loop through a list of elements, check if it already exists in db, and create if it does not exist, as follows:

for item in list:
    try:
        MyModel.objects.get(field=item)
    except MyModel.DoesNotExist:
        MyModel.objects.create(field=item)

I am expecting this to create an item in the db only if it does not exist, but this approach creates multiple items with the same field value. There seems to be some sort of caching going on around here.

I have two questions here;

How can I change this behavior in order to check for existence from the latest state of the db in each run through the loop?

Is this behavior related to me running this through a script? If the same loop would be to run within a view. would the behavior be any different?

EDIT:

I've come through suggestions to a similar question to change transaction-isolation to READ-COMMITTED . Would this result in performance drawbacks in regular Django view operations?

Answer 1

It seems like you are encountering a race condition here. If you take a look at the code for get_or_create that Django provides for you natively it looks similar to yours

try:
   obj = Person.objects.get(first_name='John', last_name='Lennon')
except Person.DoesNotExist:
   obj = Person(first_name='John', last_name='Lennon', birthday=date(1940, 10, 9))
obj.save()

In fact the code above may also be subject to race condition and create more objects than one which the documentation also says: However, if uniqueness is not enforced at the database level for the kwargs used in a get_or_create call (see unique or unique_together), this method is prone to a race-condition which can result in multiple rows with the same parameters being inserted simultaneously.

So the solution for you is make field unique in this case.

Or if the field can't be unique I would suggest that you try out using transactions explicitly.

from django.db import transaction 

with transaction.atomic():
    # do_database_stuff