Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas read_json() fails with a simple JSON string

I'm trying to create a DataFrame object from a json string (not a file):

json_string = "[{'code': '8', 'name': 'Human'}, {'code': '11', 'name': 'Orc'}]"
df = pd.read_json(json_string)

but this approach causes the following error:

ValueError: Expected object or value

The documentation makes it sound like this would be possible:

pandas.read_json(path_or_buf=None, orient=None, typ='frame', dtype=True, convert_axes=True, convert_dates=True, keep_default_dates=True, numpy=False, precise_float=False, date_unit=None, encoding=None, lines=False, chunksize=None, compression='infer')

Convert a JSON string to pandas object

  • path_or_buf : a valid JSON string or file-like, default: None The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. For instance, a local file could be file://localhost/path/to/table.json

I've already tried calling the method with several combinations of orient and none succeeded. Any tips on how to accomplish this?

I really don't want the overhead of saving my string to a file to be able to use read_json() successfully.

like image 502
karlphillip Avatar asked Feb 03 '18 01:02

karlphillip


People also ask

How read JSON string in pandas?

If you have a JSON in a string, you can read or load this into pandas DataFrame using read_json() function. By default, JSON string should be in Dict like format {column -> {index -> value}} . This is also called column orientation. Note that orient param is used to specify the JSON string format.

How do you parse a JSON string in Python?

If you have a JSON string, you can parse it by using the json.loads() method. The result will be a Python dictionary.

Does pandas work with JSON?

Big data sets are often stored, or extracted as JSON. JSON is plain text, but has the format of an object, and is well known in the world of programming, including Pandas. In our examples we will be using a JSON file called 'data.

How do I load a JSON file into a pandas DataFrame?

Reading JSON Files using Pandas To read the files, we use read_json() function and through it, we pass the path to the JSON file we want to read. Once we do that, it returns a “DataFrame”( A table of rows and columns) that stores data.


1 Answers

You can use eval to evaluate the string to dict and then pass directly to DataFrame constructor.

>>> import pandas as pd
>>> pd.DataFrame(eval("[{'code': '8', 'name': 'Human'}, {'code': '11', 'name': 'Orc'}]"))

    code   name
0   8      Human
1   11     Orc

Similarly you can also use ast.literal_eval in place of eval

>>> import ast
>>> pd.DataFrame(ast.literal_eval("[{'code': '8', 'name': 'Human'}, {'code': '11', 'name': 'Orc'}]"))

    code   name
0   8      Human
1   11     Orc
like image 82
Sohaib Farooqi Avatar answered Sep 28 '22 11:09

Sohaib Farooqi