UnicodeEncodeError: 'ascii' codec can't encode character

Question

When uploading files with non-ASCII characters I get UnicodeEncodeError:

Exception Type: UnicodeEncodeError at /admin/studio/newsitem/add/ Exception Value: 'ascii' codec can't encode character u'\xf8' in position 78: ordinal not in range(128) 

See full stack trace.

I run Django 1.2 with MySQL and nginx and FastCGI.

This is a problem that is fixed according to the Django Trac database, but I still have the problem. Any suggestions on how to fix are welcome.

EDIT: This is my image field:

image = models.ImageField(_('image'), upload_to='uploads/images', max_length=100) 

Answer 1

For anyone encountering this problem when running Django with Supervisor, the solution is to add e.g. the following to the supervisord section of Supervisor's configuration:

environment=LANG="en_US.utf8", LC_ALL="en_US.UTF-8", LC_LANG="en_US.UTF-8" 

This solved the problem for me in Supervisor 3.0a8 running on Debian Squeeze.

Also make sure Supervisor re-reads the configuration by running:

supervisorctl reread supervisorctl restart myservice 

(thanks @Udi)

---

For upstart, add in your /etc/init/myservice.conf:

env LANG="en_US.utf8" env LC_ALL="en_US.UTF-8" env LC_LANG="en_US.UTF-8"` 

(thanks @Andrii Zarubin; see Environment Variables in Upstart documentation for more information)

Answer 2

In situations where you must display a unicode string in a place that only accepts ascii (like the console or as a path) you must tell Python that you want it to replace the non ascii characters best effort.

>> problem_str = u'This is not all ascii\xf8 man' >> safe_str = problem_str.encode('ascii', 'ignore') >> safe_str 'This is not all ascii man' 

Encoding issues are prevented in the admin by the cautious handing of Django templating, but if you have ever added custom columns and forgotten to convert the values to ascii, or you override the str method for a model and forget to do this, you will get the same error, preventing template rendering.

If this string were saved into your (hopefully utf8) database there would be no problem, it looks like you are trying to upload a file that uses the title of an entity that has a non ascii character.