Convert stringified list of dictionaries back to a list of dictionaries

Question

I know that to convert a dictionary to/from a string, I use json.loads and json.dumps. However, those methods fail when given a string representing a list of dictionaries. For example,

sample_entry = [
    {"type": "test", "topic": "obama", "interval": "daily"},
    {"type": "test", "topic": "biden", "interval": "immediate"},
]

Converting that to a string with str() gives us

"[{'topic': 'obama', 'interval': 'daily', 'type': 'test'}, {'topic': 'biden', 'interval': 'immediate', 'type': 'test'}]"

Is there an easy way of converting this back to a list of dictionaries? eval does precisely what I want, but I'm concerned about running it on a user-supplied string.

Is there another way of accomplishing this that will be robust in the face of varying list lengths and variable white space?

EDIT: While json.loads(json.dumps(sample_entry)) does indeed work, json.loads(str(sample_entry)) does not. The difference, it seems, is where single quotes and double quotes are used. The data source I'm receiving this string from is unlikely to be consistent in use of ' vs ", so I'd like to account for both.

Answer 1

You may use ast.literal_eval:

>>> import ast
>>> my_str = "[{'topic': 'obama', 'interval': 'daily', 'type': 'test'}, {'topic': 'biden', 'interval': 'immediate', 'type': 'test'}]"

>>> ast.literal_eval(my_str)
[{'interval': 'daily', 'type': 'test', 'topic': 'obama'}, {'interval': 'immediate', 'type': 'test', 'topic': 'biden'}]

As per the ast.literal_eval document, it:

Safely evaluate an expression node or a Unicode or Latin-1 encoded string containing a Python literal or container display. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.

Answer 2

Converting that to a string with str() gives us...

There's your problem right there. You shouldn't be using str to serialize data. There's no guarantee that the resulting string will be something that can be recovered back into the original object.

Instead of using str(), use json.dumps. I know you said you tried it and it didn't work, but it's quite robust when used on built-in objects, so I suspect you may have just accidentally made a typo. The example dict you give serializes perfectly:

>>> sample_entry = [
...     {"type": "test", "topic": "obama", "interval": "daily"},
...     {"type": "test", "topic": "biden", "interval": "immediate"},
... ]
>>>
>>> s = json.dumps(sample_entry)
>>> print(s)
[{"interval": "daily", "topic": "obama", "type": "test"}, {"interval": "immediate", "topic": "biden", "type": "test"}]
>>>
>>> d = json.loads(s)
>>> d
[{'interval': 'daily', 'topic': 'obama', 'type': 'test'}, {'interval': 'immediate', 'topic': 'biden', 'type': 'test'}]