Why should json.loads be preferred to ast.literal_eval for parsing JSON?

Question

I have a dictionary that is stored in a db field as a string. I am trying to parse it into a dict, but json.loads gives me an error.

Why does json.loads fail on this and ast.literal_eval works? Is one preferable over the other?

>>> c.iframe_data
u"{u'person': u'Annabelle!', u'csrfmiddlewaretoken': u'wTE9RZGvjCh9RCL00pLloxOYZItQ98JN'}"

# json fails
>>> json.loads(c.iframe_data)
Traceback (most recent call last):
ValueError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

# ast.literal_eval works
>>> ast.literal_eval(c.iframe_data)
{u'person': u'Annabelle!', u'csrfmiddlewaretoken': u'wTE9RZGvjCh9RCL00pLloxOYZItQ98JN'}

Answer 1

json.loads failed because your c.iframe_data value is not a valid JSON document. In valid json document string are quoted in double quote and there isn't anything like u for converting strings to unicode.

Using json.loads(c.iframe_data) means deserialize the JSON document in c.iframe_data

ast.literal_eval is used whenever you need eval to evaluate input expression. If you have Python expressions as an input that you want to evaluate.

Is one preferable over the other?

It depends on the data. See this answer for more context.

Answer 2

I have a dictionary that is stored in a db field as a string.

This is a design fault. While it's perfectly possible, as someone appears to have done, to extract the repr of a dictionary, there's no guarantee that the repr of an object can be evaluated at all.

In the presence of only string keys and string and numeric values, most times the Python eval function will reproduce the value from its repr, but I am unsure why you think that this would make it valid JSON, for example.

I am trying to parse it into a dict, but json.loads gives me an error.

Naturally. You aren't storing JSON in the database, so it hardly seems reasonable to expect it to parse as JSON. While it's interesting that ast.literal_eval can be used to parse the value, again there are no guarantees beyond relatively simple Python types.

Since it appears your data is indeed limited to such types, the real solution to your problem is to correct the way the data is stored, by converting the dictionary to a string with json.dumps before storage in the database. Some database systems (e.g., PostgreSQL) have JSON types to make querying such data simpler, and I'd recommend you use such types if they are available to you.

As to which is "better," that will always depend on the specific application, but JSON was explicitly designed as a compact human-readable machine-parseable format for simple structured data, whereas your current representation is based on formats specific to Python, which (for example) would be tediously difficult to evaluate in other languages. JSON is the applicable standard here, and you will benefit from using it.