Django + Postgres: A string literal cannot contain NUL (0x00) characters

Question

I'm syncing a large amount of data and I'm getting this error back: A string literal cannot contain NUL (0x00) characters. Obviously this is a postgres problem, but I'm not quite sure how to solve it. Is there a way to strip null characters out at the Django model level? I have a large set of fields that I'm syncing.

Answer 1

This bug fix suggests:

s.decode("utf-8", errors="replace").replace("\x00", "\uFFFD")

Only the .replace is necessary for the OP's issue, which replaces the null with a � character. I've included .decode too as it protects against other encoding issues that might arise in similar situations.

This would go in a .clean method somewhere - maybe subclass TextField or CharField if you want to apply it globally.

Answer 2

Unless you definitely do want to store NUL characters, you should sanitize your text so it does not contain them. At the model level, you'd define a clean_fieldname method to do that.

If you do want to store them, you need to store them in a binary-compatible field in the database. Django 1.6+ has BinaryField which should work.