UnicodeDecodeError after upgrading to django-rest-framework 3

Question

Under django-rest-framework 2, the following works fine:

from rest_framework import rest_response, generics
from rest_framework.renderers import JSONRenderer, BrowsableAPIRenderer

class SomeView(generics.GenericAPIView):
    renderer_classes = JSONRenderer, BrowsableAPIRenderer

    def get(self, request, *args, **kwargs):
        ... 
        # Build a response dict with non-ascii in it
        ...
        return rest_response.Response(some_dict_with_non_ascii_in_it_somewhere)

I didn't have to explicitly encode any non-ascii...

However, after upgrading to DRF 3, the same code now throws the following error:

Traceback (most recent call last):
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 63, in __call__
    return self.application(environ, start_response)
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/whitenoise/base.py", line 119, in __call__
    return self.application(environ, start_response)
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 189, in __call__
    response = self.get_response(request)
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/django/core/handlers/base.py", line 218, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/django/core/handlers/base.py", line 261, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/django_extensions/management/technical_response.py", line 5, in null_technical_500_response
    six.reraise(exc_type, exc_value, tb)
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/django/core/handlers/base.py", line 164, in get_response
    response = response.render()
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/django/template/response.py", line 158, in render
    self.content = self.rendered_content
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/rest_framework/response.py", line 71, in rendered_content
    ret = renderer.render(self.data, media_type, context)
  File "/Users/troy/.virtualenvs/publisher/lib/python2.7/site-packages/rest_framework/renderers.py", line 104, in render
    separators=separators
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 210, in encode
    return ''.join(chunks)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 671: ordinal not in range(128)

I'm guessing DRF 3 now has some new config value, somewhere, that was default under DRF 2. I've tried setting the REST_FRAMEWORK UNICODE_JSON setting to True, but I still get the same error...

Is there a setting that can make that piece behave as DRF 2 did? or does DRF 3 need me to hunt down the non-ascii character in my dictionary and manually encode it?

Answer 1

I found the answer.

In DRF 2, rest_framework.JSONRenderer.ensure_ascii is set to True. In DRF 3, rest_framework.JSONRenderer.ensure_ascii is set to not api_settings.UNICODE_JSON (I had missed the not there earlier, when I wrote the question...).

So to get it to behave like DRF 2, I needed to set the 'UNICODE_JSON' to False instead of True, like I had tried before (it is True by default):

REST_FRAMEWORK = {
    ...
    'UNICODE_JSON': False
}

Alternatively, I could, of course, encode my dictionary values, which in many cases could be the better option.

Answer 2

By default Python 2.7 consider strings are binary. Try adding at the top of your files:

from __future__ import unicode_literals

That will make your strings unicode by default and should help get them converted correctly.