Is it safe to do a data migration as just one operation in a larger Django migration?

Question

I am handling what I assume is a common issue: I've realized that an existing model field of the model Foo would be better as a completely seperate model Bar with a foreign key to Foo. So, we need to make a schema migration. But what's more, since there is already existing data in the model field of Foo, we need to make a data migration before we delete that field.

So we have identified that there are three distinct steps to take:

1. Create the new table Bar
2. Migrate the existing data in Foo to the new table Bar
3. Delete the existing field in Foo

First, I make all the needed model changes in models.py, and then auto-generate a migration. Everything looks good, except we're going to lose all the data in the field, so I need to add one extra operation to handle the data migration (RunPython). I would end up with something like the following:

def do_data_migration(apps, schema_editor):
    # Migrate data from Foo to Bar

class Migration(migrations.Migration):

    dependencies = [ 
        ('exampleapp', 'migration_003'),
    ]   

    operations = [ 
        migrations.CreateModel(
            # Create the new model Bar
        ),  
        migrations.AddField(
            # Add the foreign key field to model Foo
        ),  
        migrations.RunPython(
            do_data_migration
        ),
        migrations.RemoveField(
            # Remove the old field from Foo
        ),  
    ]

Is it safe to run a data migration as one of several operations in a migration? My worries are that there is any sort of locking going on, or if perhaps the app registry that RunPython passes to do_data_migration won't be up to date with the preceding operations?

I am aware the I could create three migrations: one for CreateModel and AddField, the second for RunPython, and the last for RemoveField. The question is if it is functionally equivalent to do all four steps in a single migration (which provides the added benefit of making the entire migration easier to understand.)

Answer 1

With regards to Django itself, this is perfectly safe. Each operation will receive the correct state based on all previous migrations and operations within the same migration. Your RunPython operation will receive an app registry that includes the new Bar model and still has the old field on Foo.

What may not be safe is the database-side of the operation. If a database supports DDL (Data Definition Language) in transactions, Django will run the complete migration in a single transaction. PostgreSQL, for example, supports DDL in transactions, but does not allow you to mix schema changes and data changes in the same transaction. Attempting to do both within a single migration/transaction will result in an error.

If you use MySQL or Oracle, which do not support DDL transactions and will only run the RunPython operation in a transaction, you can safely put all operations in the same migration. However, you will lose out on some cross-database compatibility.