Problem loading Django fixture: IntegrityError: (1062, "Duplicate entry '4' for key 'user_id'")

Question

I used the following commands to generate 2 fixtures:

./manage.py dumpdata --format=json --indent=4 --natural auth.User > fixtures/user.json
./manage.py dumpdata --format=json --indent=4 --natural --exclude=contenttypes --exclude=auth > fixtures/full.json

I've got the following fixture named user.json:

[
    {
        "pk": 4, 
        "model": "auth.user", 
        "fields": {
            "username": "foo", 
            "first_name": "Se\u00e1n", 
            "last_name": "Hayes", 
            "is_active": true, 
            "is_superuser": true, 
            "is_staff": true, 
            "last_login": "2010-09-27 21:57:45", 
            "groups": [], 
            "user_permissions": [], 
            "password": "!", 
            "email": "[email protected]", 
            "date_joined": "2010-09-27 21:57:45"
        }
    }
]

and the following fixture named full.json:

[
    {
        "pk": "72a75887b4a0ce06a61f9183fe1c0e15", 
        "model": "sessions.session", 
        "fields": {
            "expire_date": "2010-10-11 21:57:45", 
            "session_data": "gAJ9cQEoVRJfYXV0aF91c2VyX2JhY2tlbmRxAlUOZmIuYXV0aC5GYkF1dGhxA1UNX2F1dGhfdXNl\ncl9pZHEEigEEdS5hOGZlODU0MmRjYmUwNmEzODIwNjhiYzYyODc2MWQxZA==\n"
        }
    }, 
    {
        "pk": 1, 
        "model": "sites.site", 
        "fields": {
            "domain": "example.com", 
            "name": "example.com"
        }
    }, 
    {
        "pk": 2, 
        "model": "common.userprofile", 
        "fields": {
            "money": 10, 
            "energy": 10, 
            "experience": 0, 
            "stamina": 10, 
            "health": 10, 
            "user": 4
        }
    }, 
    {
        "pk": 2, 
        "model": "missions.missionprofile", 
        "fields": {
            "user": 4, 
            "last_area_viewed": null
        }
    }, 
    {
        "pk": 1, 
        "model": "fb.facebookuser", 
        "fields": {
            "updated": "2010-09-27 21:57:45", 
            "uid": "24411841", 
            "created": "2010-09-27 21:57:45", 
            "access_token": "foo", 
            "url": "http://www.facebook.com/profile.php?id=24411841", 
            "user": 4, 
            "img_url": null, 
            "name": "Se\u00e1n Hayes"
        }
    }
]

Running the following commands (in either order):

./manage.py loaddata user
./manage.py loaddata full

raises the following exception:

Problem installing fixture '/projectpath/fixtures/full.json': Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/django/core/management/commands/loaddata.py", line 169, in handle
    obj.save(using=using)
  File "/usr/lib/pymodules/python2.6/django/core/serializers/base.py", line 165, in save
    models.Model.save_base(self.object, using=using, raw=True)
  File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 528, in save_base
    result = manager._insert(values, return_id=update_pk, using=using)
  File "/usr/lib/pymodules/python2.6/django/db/models/manager.py", line 195, in _insert
    return insert_query(self.model, values, **kwargs)
  File "/usr/lib/pymodules/python2.6/django/db/models/query.py", line 1479, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/lib/pymodules/python2.6/django/db/models/sql/compiler.py", line 783, in execute_sql
    cursor = super(SQLInsertCompiler, self).execute_sql(None)
  File "/usr/lib/pymodules/python2.6/django/db/models/sql/compiler.py", line 727, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/pymodules/python2.6/django/db/backends/util.py", line 15, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/pymodules/python2.6/django/db/backends/mysql/base.py", line 86, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 166, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/pymodules/python2.6/MySQLdb/connections.py", line 35, in defaulterrorhandler
    raise errorclass, errorvalue
IntegrityError: (1062, "Duplicate entry '4' for key 'user_id'")

Any ideas on what's going on? I'm aware there's a problem with forward references in MySQL, but that shouldn't matter if the User object with id 4 is being installed before the fixtures that contain a foreign key to it, right?

Answer 1

As per Ashok's comment, when I had the same problem, it was solved by changing my signal handler to check for whether it is running in "raw" mode which apparently means a fixture is being loaded:

def create_user_profile(sender, instance, created, **kwargs):
    if created and not kwargs.get('raw', False): 
        UserProfile.objects.create(user=instance) 

See How do I prevent fixtures from conflicting with django post_save signal code?

Answer 2

Static fixture of Django is an ANTI-PATTERN, that is the reason you are having this kind of problem. I suggest you to use some library that generate these fixtures for you, including the ID naturally.

I suggest you Dynamic Dynamic Fixture