Django ORM, CharField and blank=True

Question

Django's documentation is quite clear about storing empty strings as "" rather than NULL at a database level (so there is only one possible format for empty data):

Note that empty string values will always get stored as empty strings, not as NULL. Only use null=True for non-string fields such as integers, booleans and dates. For both types of fields, you will also need to set blank=True if you wish to permit empty values in forms, as the null parameter only affects database storage (see blank).

Nonetheless, after adding a new field, I've started encountering IntegrityErrors on the new field (phone_number).

null value in column "phone_number" violates non-null constraint

That model looks like this with the new field (I performed a migration via south):

class Person(models.Model):
    user = models.ForeignKey(User)
    description = models.TextField(blank=True)
    phone_number = models.CharField(blank=True)

I've since (temporarily) resolved the issue by setting null=True on phone_number, but now I have hundreds of entries with empty strings, and a single NULL value in my database. (I also tried adding default='' to the phone_number field, but I was still seeing IntegrityError issues.)

In the past I've always used MySQL, but on this project I'm using Postgres. The generated SQL insert attempt is:

'INSERT INTO "people_person" ("user_id", "description", "gender", "birthdate", "default_image_id", "zip_code", "beta_status") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "people_person"."id"'.

My expectation would be that Django would be inserting a blank string into the "phone_number" column, but it doesn't appear to be doing so. The other thing I might expect would be Django to include a SET DEFAULT in the CREATE TABLE statement, but it doesn't. So Postgres gets angry about the NOT NULL on that column.

Thanks!

Answer 1

As is usually the case with problems that are so seemingly intractable, the issue at hand was user error.

My application had two entry points - two WSGI files, but only one code base. Normally, Apache will only reload your code if the file is touched. My deploy script was only touching one of those WSGI files - which meant that people reaching my site via the other WSGI file were still seeing old code. Worse, the database was modified under that old code, but the models were still as they were before.

This in turn caused the IntegrityError issues. Django didn't know about the phone_number field, so even though I had set blank=True, Django made no effort to insert a blank value - and the database of course thought that meant NULL.

This caused a series of different to track down errors, including the above error.

It's amazing how often really tough issues like these are caused by dumb minor omissions - like a deploy script I wrote 2 months ago and forgot to update.

Thanks for reading folks, I've upvoted the other answers, but I need to accept mine since it was ultimately the solution.

Answer 2

I discovered that if you explicitly set the field value to None you will still get these errors. In other words the default= thing is applied as soon as you create the python object, rather then when you save it to the database.

I guess that is reasonable but it was a bit unexpected.