Need a way to load embedded, escaped JSON strings in Python

Question

I have to parse the following JSON string:

{"JobDescription":"{\"project\": \"1322\", \"vault\": \"qa-122\"}"}'

If I try to use json.loads, I get the following:

>>> import json
>>> print json.loads('{"JobDescription":"{\"project\": \"1322\", \"vault\": \"qa-122\"}"}')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
 File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
 File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 381, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 1 column 22 (char 21)

I don't have any control over the string I receive as its generated by another system.

Answer 1

You are not producing embedded backslashes; Python is interpreting the \" as an escaped quote and the final string just contains the quote:

>>> '{"JobDescription":"{\"project\": \"1322\", \"vault\": \"qa-122\"}"}'
'{"JobDescription":"{"project": "1322", "vault": "qa-122"}"}'

Use a raw string or double the slashes:

>>> r'{"JobDescription":"{\"project\": \"1322\", \"vault\": \"qa-122\"}"}'
'{"JobDescription":"{\\"project\\": \\"1322\\", \\"vault\\": \\"qa-122\\"}"}'
>>> '{"JobDescription":"{\\"project\\": \\"1322\\", \\"vault\\": \\"qa-122\\"}"}'
'{"JobDescription":"{\\"project\\": \\"1322\\", \\"vault\\": \\"qa-122\\"}"}'

This then loads fine:

>>> import json
>>> json.loads('{"JobDescription":"{\\"project\\": \\"1322\\", \\"vault\\": \\"qa-122\\"}"}')
{'JobDescription': '{"project": "1322", "vault": "qa-122"}'}

and you can decode the nested JSON document from there:

>>> decoded = json.loads('{"JobDescription":"{\\"project\\": \\"1322\\", \\"vault\\": \\"qa-122\\"}"}')
>>> json.loads(decoded['JobDescription'])
{'project': '1322', 'vault': 'qa-122'}