Django: Prevent string being unicode escaped? [duplicate]

Question

I'm trying to prevent a string (in this case the value variable) in a POST request being escaped as it's to be stored in JSON. My code is

def addProduct(request):
    if request.POST:
        post = {}
        for key in request.POST:
            value = request.POST[key].encode('utf-8')
            try:
                value = json.loads(value).encode('utf-8')
            except Exception:
                pass
            post[key] = value.encode('utf-8')
        doc = json.dumps(post)

Debugging I can see value is of type unicode which I believe is how Django handles request objects. The actual string although unicode doesn't get its special characters escaped until post[key] = value. If I try to change this to post[key] = value.encode('utf-8') to prevent it getting escaped I get the error: 'ascii' codec can't decode byte 0xe2 in position 38: ordinal not in range(128)

Any ideas?

Answer 1

If you want json.dumps to maintain the special characters I think you may find useful the arguments ensure_ascii=False.

1. Take a look at this answer: Unicode values in strings are escaped when dumping to JSON in Python
2. This is the docs for json.dumps

Instead of doing it yourself, ensure_ascii=False I think will solve the problem of json escaping the output.

Ex:

json.dumps({'h':u'\xc2\xa3'},ensure_ascii=False)
>>>u'{"h": "\xc2\xa3"}'

UPDATE: Comparison of json.dumps with and without ensure_ascii and a unicode string:

In [7]: json.dumps({'a':u'\u00a3'},ensure_ascii=False)
Out[7]: u'{"a": "\xa3"}'

In [8]: json.dumps({'a':u'\u00a3'})
Out[8]: '{"a": "\\u00a3"}'

Hope this helps!