What value do I use for _ptr when migrating reparented classes with South?

Question

I have two classes, one of which is descended from the other, and I would like to make them both sibling classes descended from the same base class.

Before:

from django.db import models

class A(models.Model):
    name = models.CharField(max_length=10)

class B(models.Model):
    title = models.CharField(max_length=10)

After:

from django.db import models

class Base(models.Model):
    name = models.CharField(max_length=10)

class A(Base):
    pass

class B(Base):
    title = models.CharField(max_length=10)

When I generate a schema migration, this is the output, including my answers to the questions:

+ Added model basetest.Base
? The field 'B.a_ptr' does not have a default specified, yet is NOT NULL.
? Since you are removing this field, you MUST specify a default
? value to use for existing rows. Would you like to:
?  1. Quit now, and add a default to the field in models.py
?  2. Specify a one-off value to use for existing columns now
?  3. Disable the backwards migration by raising an exception.
? Please select a choice: 3
- Deleted field a_ptr on basetest.B
? The field 'B.base_ptr' does not have a default specified, yet is NOT NULL.
? Since you are adding this field, you MUST specify a default
? value to use for existing rows. Would you like to:
?  1. Quit now, and add a default to the field in models.py
?  2. Specify a one-off value to use for existing columns now
? Please select a choice: 2
? Please enter Python code for your one-off default value.
? The datetime module is available, so you can do e.g. datetime.date.today()
>>> 37
+ Added field base_ptr on basetest.B
? The field 'A.id' does not have a default specified, yet is NOT NULL.
? Since you are removing this field, you MUST specify a default
? value to use for existing rows. Would you like to:
?  1. Quit now, and add a default to the field in models.py
?  2. Specify a one-off value to use for existing columns now
?  3. Disable the backwards migration by raising an exception.
? Please select a choice: 3
- Deleted field id on basetest.A
? The field 'A.name' does not have a default specified, yet is NOT NULL.
? Since you are removing this field, you MUST specify a default
? value to use for existing rows. Would you like to:
?  1. Quit now, and add a default to the field in models.py
?  2. Specify a one-off value to use for existing columns now
?  3. Disable the backwards migration by raising an exception.
? Please select a choice: 3
- Deleted field name on basetest.A
? The field 'A.base_ptr' does not have a default specified, yet is NOT NULL.
? Since you are adding this field, you MUST specify a default
? value to use for existing rows. Would you like to:
?  1. Quit now, and add a default to the field in models.py
?  2. Specify a one-off value to use for existing columns now
? Please select a choice: 2
? Please enter Python code for your one-off default value.
? The datetime module is available, so you can do e.g. datetime.date.today()
>>> 73
+ Added field base_ptr on basetest.A
Created 0002_auto__add_base__del_field_b_a_ptr__add_field_b_base_ptr__del_field_a_i.py. You can now apply this migration with: ./manage.py migrate basetest

I do not know how to answer the questions about default values for B.base_ptr and A.base_ptr. Any constant I give causes the migration to fail when it is run, with this output:

FATAL ERROR - The following SQL query failed: CREATE TABLE "_south_new_basetest_a" ()
The error was: near ")": syntax error
RuntimeError: Cannot reverse this migration. 'B.a_ptr' and its values cannot be restored.

This is the result when I use sqlite3, by the way. Using Postgres gives something like this:

FATAL ERROR - The following SQL query failed: ALTER TABLE "basetest_a" ADD COLUMN "base_ptr_id" integer NOT NULL PRIMARY KEY DEFAULT 73;
The error was: could not create unique index "basetest_a_pkey"
DETAIL:  Key (base_ptr_id)=(73) is duplicated.

Error in migration: basetest:0002_auto__add_base__del_field_b_a_ptr__add_field_b_base_ptr__del_field_a_i
IntegrityError: could not create unique index "basetest_a_pkey"
DETAIL:  Key (base_ptr_id)=(73) is duplicated.

What values should I use for base_ptr to make this migration work? Thanks!

Answer 1

If base will not be instantiated on its own, you can easily solve the problem using abstract = True prop to class Meta.

Example code:

from django.db import models

class Base(models.Model):
    name = models.CharField(max_length=10)
    class Meta:
        abstract = True

class A(Base):
    pass

class B(Base):
    title = models.CharField(max_length=10)

Answer 2

You do this in separate phases.

Phase 1: Create your "Base" model in the code. On the A and B models, add base_ptr as a nullable FK to Base (the name base_ptr is made by lowercasing the class-name Base, adapt your names accordingly). Specify db_column='base_ptr' on the new column, so you don't get an _id suffix added. Don't change parenthood yet: Keep B as a child of A and A as it was before (Base has no child classes yet). Add a migration to make the respective database changes, and run it.

Phase 2: Create a data migration, copying relevant data around. You should probably copy all A data into Base, remove redundant A records (those that served B instances), and in remaining records (of both A and B) copy the id into base_ptr. Note that the child class B uses two tables -- its id field comes from A's table, and on its own table there is a field a_ptr which is a FK to A -- so your update operation will be more efficient if you copy values from a_ptr to base_ptr. Make sure the copying into base_ptr occurs after the copying into the Base table, so you don't violate the FK constraints.

Phase 3: Now change the models again -- remove the explicit base_ptr FK and change parents to the way you like, and create a third migration (automatic schema migration). Note that setting the parent to Base implicitly defines a non-nullable base_ptr field, so with respect to the base_ptr fields, you are only changing a nullable field into non-nullable, and no default is needed.

You should still be asked for a default value for a_ptr -- the implicit FK from B to A that is removed when the parent is changed from A to Base; the default is needed for the migration in the backward direction. You can either do something that will fail the backward migration, or, if you do want to support it, add an explicit nullable a_ptr to B, like the base_ptr columns you used before. This nullable column can then be removed in a fourth migration.