Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas read_json: "If using all scalar values, you must pass an index"

I have some difficulty in importing a JSON file with pandas.

import pandas as pd map_index_to_word = pd.read_json('people_wiki_map_index_to_word.json') 

This is the error that I get:

ValueError: If using all scalar values, you must pass an index 

The file structure is simplified like this:

{"biennials": 522004, "lb915": 116290, "shatzky": 127647, "woode": 174106, "damfunk": 133206, "nualart": 153444, "hatefillot": 164111, "missionborn": 261765, "yeardescribed": 161075, "theoryhe": 521685} 

It is from the machine learning course of University of Washington on Coursera. You can find the file here.

like image 621
Marco Fumagalli Avatar asked Jul 14 '16 17:07

Marco Fumagalli


People also ask

When using scalar values you must pass an index?

The most common ways of creating data frames in Python are using lists and dictionaries. You can use a list of lists or a dictionary of lists.

Can we create DataFrame in pandas using a scalar value?

Constructing pandas DataFrame from values in variables gives "ValueError: If using all scalar values, you must pass an index"

What is scalar values in pandas?

Scalars are single values representing one unit of data, such as an integer or bool , as opposed to data structures like a list or tuple , which are composed of scalars.

How do you pass an index into a DataFrame?

Python range as the index of the DataFrame First, we will create a Python list then pass it to the pd. Index() function which returns the DataFrame index object. Then we pass the returned DataFrame index object to the set_index() function to set it as the new index of the DataFrame.


2 Answers

Try

ser = pd.read_json('people_wiki_map_index_to_word.json', typ='series') 

That file only contains key value pairs where values are scalars. You can convert it to a dataframe with ser.to_frame('count').

You can also do something like this:

import json with open('people_wiki_map_index_to_word.json', 'r') as f:     data = json.load(f) 

Now data is a dictionary. You can pass it to a dataframe constructor like this:

df = pd.DataFrame({'count': data}) 
like image 121
ayhan Avatar answered Sep 27 '22 23:09

ayhan


You can do as @ayhan mention which will give you a column base format

Method 1

Or you can enclose the object in [ ] (source) as shown below to give you a row format that will be convenient if you are loading multiple values and planing on using matrix for your machine learning models.

df = pd.DataFrame([data]) 

Method 2

like image 23
Adonis H. Avatar answered Sep 28 '22 00:09

Adonis H.