How to organize migration for two related models and automatically set default field value for id of newly created object?

Question

Suppose there is a production database, there is some data in it. I need to migrate in the next tricky case.

There is a model (already in db), say Model, it has foreign keys to other models.

class ModelA: ...
class ModelX: ...

class Model:
  a = models.ForeignKey(ModelA, default = A)
  x = models.ForeignKey(ModelX, default = X)

And we need to create one more model ModelY to which Model should refer. And when creating a Model, an object should have some default value related to some ModelY object, which is obviously not yet available, but we should create it during migration.

class ModelY: ...
class Model:
  y = models.ForeignKey (ModelY, default = ??????)

So the migration sequence should be:

• Create ModelY table
• Create a default object in this table, put its id somewhere
• Create a new field y in the Model table, with the default value taken from the previous paragraph

And I'd like to automate all of this, of course. So to avoid necessity to apply one migration by hands, then create some object, then write down it's id and then use this id as default value for new field, and only then apply another migration with this new field.

And I'd also like to do it all in one step, so define both ModelY and a new field y in the old model, generate migration, fix it somehow, and then apply at once and make it work.

Are there any best practices for such case? In particular, where to store this newly created object's id? Some dedicated table in same db?

Answer 1

You won't be able to do this in a single migration file, however you'll be able to create several migrations files to achieve this. I'll have a go at helping you out though I'm not totally certain this is what you want, it should teach you a thing or two about Django migrations.

I'm going to refer to two types of migrations here, one is a schema migration, and these are the migration files you typically generate after changing your models. The other is a data migration, and these need to be created using the --empty option of the makemigrations command, e.g. python manage.py makemigrations my_app --empty, and are used to move data around, set data on null columns that need to be changed to non-null, etc.

class ModelY(models.Model):
    # Fields ...
    is_default = models.BooleanField(default=False, help_text="Will be specified true by the data migration")

class Model(models.Model):
    # Fields ...
    y = models.ForeignKey(ModelY, null=True, default=None)

You'll notice that y accepts null, we can change this later, for now you can run python manage.py makemigrations to generate the schema migration.

To generate your first data migration run the command python manage.py makemigrations <app_name> --empty. You'll see an empty migration file in your migrations folder. You should add two methods, one that is going to create your default ModelY instance and assign it to your existing Model instances, and another that will be a stub method so Django will let you reverse your migrations later if needed.

from __future__ import unicode_literals

from django.db import migrations


def migrate_model_y(apps, schema_editor):
    """Create a default ModelY instance, and apply this to all our existing models"""
    ModelY = apps.get_model("my_app", "ModelY")
    default_model_y = ModelY.objects.create(something="something", is_default=True)

    Model = apps.get_model("my_app", "Model")
    models = Model.objects.all()
    for model in models:
        model.y = default_model_y
        model.save()


def reverse_migrate_model_y(apps, schema_editor):
    """This is necessary to reverse migrations later, if we need to"""
    return


class Migration(migrations.Migration):

    dependencies = [("my_app", "0100_auto_1092839172498")]

    operations = [
        migrations.RunPython(
            migrate_model_y, reverse_code=reverse_migrate_model_y
        )
    ]

Do not directly import your models to this migration! The models need to be returned through the apps.get_model("my_app", "my_model") method in order to get the Model as it was at this migration's point in time. If in the future you add more fields and run this migration your models fields may not match the databases columns (because the model is from the future, sort of...), and you could receive some errors about missing columns in the database and such. Also be wary of using custom methods on your models/managers in migrations because you won't have access to them from this proxy Model, usually I may duplicate some code to a migration so it always runs the same.

Now we can go back and modify the Model model to ensure y is not null and that it picks up the default ModelY instance in the future:

def get_default_model_y():
    default_model_y = ModelY.objects.filter(is_default=True).first()
    assert default_model_y is not None, "There is no default ModelY to populate with!!!"
    return default_model_y.pk  # We must return the primary key used by the relation, not the instance

class Model(models.Model):
    # Fields ...
    y = models.ForeignKey(ModelY, default=get_default_model_y)

Now you should run python manage.py makemigrations again to create another schema migration.

You shouldn't mix schema migrations and data migrations, because of the way migrations are wrapped in transactions it can cause database errors which will complain about trying to create/alter tables and execute INSERT queries in a transaction.

Finally you can run python manage.py migrate and it should create a default ModelY object, add it to a ForeignKey of your Model, and remove the null to make it like a default ForeignKey.