django db migration failed with postgres

Question

I made some changes to my models and then ran a

python manage.py makemigrations python manage.py migrate

and I got this traceback:

Operations to perform:
  Synchronize unmigrated apps: staticfiles, messages
  Apply all migrations: sessions, admin, study, auth, quiz, contenttypes, main
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying quiz.0013_auto_20151005_0644...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 393, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 444, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 154, in apply_migration
    self.recorder.record_applied(migration.app_label, migration.name)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/recorder.py", line 67, in record_applied
    self.migration_qs.create(app=app, name=name)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 348, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 734, in save
    force_update=force_update, update_fields=update_fields)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 762, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 846, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 885, in _do_insert
    using=using, raw=raw)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 920, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 974, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "django_migrations_pkey"
DETAIL:  Key (id)=(27) already exists.

I didn't know quite how to interpret the error that the primary key already exists. Strangely enough it added the field into the database because I was seeing it output on my site and when I ran a shell I could see that the fields were added and the default data 'that I chose' was populated.

I decided to run the migration again to see if it would pass after a second time and I got a different traceback...

   Operations to perform:
  Synchronize unmigrated apps: staticfiles, messages
  Apply all migrations: sessions, admin, study, auth, quiz, contenttypes, main
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying quiz.0013_auto_20151005_0644...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 393, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 444, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 148, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 62, in database_forwards
    field,
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 398, in add_field
    self.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 111, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "lang1_back_to_choice" of relation "quiz_langpairinstructions" already exists

Now if I try it again I will always get the second traceback. I am wondering what I should do so I can make migrations in the futures. Why am I getting this error if everything appears to have gone smoothly in the db itself?

Answer 1

There is another way. Since you know the last id value in django_migrations table, say it is 100. Then, you need to do this:

ALTER SEQUENCE django_migrations_id_seq RESTART WITH 101;

Answer 2

I overlooked the name of the 'django_migrations_pkey' column and I didn't realize that was in the database. Somehow django was one step behind the actual database and trying to use the primary key (pk=27) instead of the number it was actually on (which should have been 28)

I was unsure about how to change this from within django so instead I just deleted the last column from the database so it was one primary key shorter and on track with django and it started working just fine.

Answer 3

I got this error after I had previously manually tinkered with the django migrations tables entries (after some previously dodgy migrations were added then removed).

This left the django_migrations IDs out of order (so to speak) with the default position of the entry in the django_migrations table.

The issue is that django takes the ID of the last entry and +1 to create a new entry (in table below: 197 + 1 = 198), however as you can see Key (id)=(198) already exists in the table.

187 | foo_app             | 0016_auto_20161220_2332                     | 2017-01-06 05:22:07.666172+00
198 | bar_app             | 0004_auto_20160423_2122                     | 2017-01-13 05:38:31.922738+00
197 | baz_app             | 0013_auto_20170203_2311                     | 2017-02-08 18:50:22.70143+00

As you can see the numbers for the primary key on the left are out of order. What I did was manually delete the out of order entries in the django_migration then re-add them.

DELETE from django_migrations where name = '0004_auto_20160423_2122';
DELETE from django_migrations where name = '0013_auto_20170203_2311';

INSERT INTO django_migrations VALUES (188, 'baz_app', '0013_auto_20170203_2311', date());
INSERT INTO django_migrations VALUES (189, 'bar_app', '0004_auto_20160423_2122', date());

Also of note is that the migration that I was initially trying to add 'new_app, '0002_auto_00000000_1111' actually modified the new_app table in postgres, however the django_migrations table didn't get updated.

So the second time I tried to re-run the migration 0002_auto_00000000_1111 I got the error:

django.db.utils.ProgrammingError: column "new_app_field" of relation "new_app_new" already exists

My solution above fixed the problem for me.