How to load unicode string into json in Python?

Question

I am trying to read file from a compressed file and convert data into json/ dictionary. But there is unicode issue that I have been struggling for a while. Can anyone help ?

exfile_obj = tar.extractfile(member)
data = exfile_obj.read()
print(type(data)) ## shows str
print(data)  ## it is something like: "{u'building': False, u'displayName': u'Tam\\xe1s Kosztol\\xe1nczi', u'changeSet': {u'items': u'comment'}}"
json_obj = json.loads(data) # it is a unicode object.

Answer 1

That data is a string representation of a Python dictionary. You can convert it to a dictionary using ast.literal_eval, and you can convert that dict to a JSON string using json.dumps.

import ast
import json

src = "{u'building': False, u'displayName': u'Tam\\xe1s Kosztol\\xe1nczi', u'changeSet': {u'items': u'comment'}}"
data = ast.literal_eval(src)
print(data)
j = json.dumps(data)
print(j)

output

{'building': False, 'displayName': 'Tamás Kosztolánczi', 'changeSet': {'items': 'comment'}}
{"building": false, "displayName": "Tam\u00e1s Kosztol\u00e1nczi", "changeSet": {"items": "comment"}}