Why does Django drop the SQL DEFAULT constraint when adding a new column?

Question

In the latest Django (2.2), when I add a new field to a model like this:

new_field= models.BooleanField(default=False)

Django runs the following commands for MySQL:

ALTER TABLE `app_mymodel` ADD COLUMN `new_field` bool DEFAULT b'0' NOT NULL;
ALTER TABLE `app_mymodel` ALTER COLUMN `new_field` DROP DEFAULT;
COMMIT;

While this works when everything is updated, this is very problematic because old versions of the application can no longer create models after this migration is run (they do not know about new_field). Why not just keep the DEFAULTconstraint?

Answer 1

Why not just keep the DEFAULT constraint?

Because Django handles the default model field option at the application level, not the database level. So the real question is why it sets the DEFAULT constraint at all.

But first: Django does not use database-level defaults. (From the documentation: "Django never sets database defaults and always applies them in the Django ORM code."). This has been true from the beginning of the project. There has always been some interest in changing it (the first issue on the subject is 14 years old), and certainly other frameworks (Rails, I think, and SQLAlchemy) have shown that it is possible.

But there are good reasons beyond backwards compatibility for handling defaults at the application level. Such as: the ability to express arbitrarily complex computations; not having to worry about subtle incompatibilities across database engines; the ability to instantiate a new instance in code and have immediate access to the default value; the ability to present the default value to users in forms; and more.

Based on the two most recent discussions on the subject, I'd say there's little appetite to incorporate database defaults into the semantics of default, but there is support for adding a new db_default option.

Now, adding a new non-nullable field to an existing database is a very different use case. In that situation, you have to provide a default to the database for it to perform the operation. makemigrations will try to infer the right value from your default option if it can, and if not it will force you to specify a value from the command line. So the DEFAULT modifier is used for this limited purpose and then removed.

As you've noticed, the lack of database-level defaults in Django can make continuous deployment harder. But the solution is fairly straightforward: just re-add the default yourself in a migration. One of the great benefits of the migrations system is that it makes it easy to make arbitrary, repeatable, testable changes to your database outside of Django's ORM. So just add a new RunSQL migration operation:

operations = [
    # Add SQL for both forward and reverse operations
    migrations.RunSQL("ALTER TABLE app_mymodel ALTER COLUMN new_field SET DEFAULT 0;",
                      "ALTER TABLE app_mymodel ALTER COLUMN new_field DROP DEFAULT;")
]

You can put that in a new migration file or simply edit the automatically generated one. Depending on your database and its support for transactional DDL, the sequence of operations may or may not be atomic.