Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to append a dictionary to a pandas dataframe?

I have a set of urls containing json files and an empty pandas dataframe with columns representing the attributes of the jsnon files. Not all json files have all the attributes in the pandas dataframe. What I need to do is to create dictionaries out of the json files and then append each dictionary to the pandas dataframe as a new row and, in case the json file doesn't have an attribute matching a column in the dataframe this has to be filled blank.

I managed to create dictionaries as:

import urllib2
import json  

url = "https://cws01.worldstores.co.uk/api/product.php?product_sku=ULST:7BIS01CF"
data = urllib2.urlopen(url).read()
data = json.loads(data)

and then I tried to create a for loop as follows:

row = -1
for i in links:
    row = row + 1
    data = urllib2.urlopen(str(i)).read()
    data = json.loads(data)
    for key in data.keys():
        for column in df.columns:
            if str(column) == str(key):
                df.loc[[str(column)],row] = data[str(key)]
            else:
                df.loc[[str(column)],row] = None

where df is the dataframe and links is the set of urls

However, I get the following error:

raise KeyError('%s not in index' % objarr[mask])

KeyError: "['2_seater_depth_mm'] not in index"

where ['2_seater_depth_mm'] is the first column of the pandas dataframe

like image 835
Blue Moon Avatar asked Jul 29 '15 08:07

Blue Moon


People also ask

How to convert a dictionary to a pandas Dataframe?

You may use the following template to convert a dictionary to Pandas DataFrame: import pandas as pd my_dict = {key:value,key:value,key:value,...} df = pd.DataFrame (list (my_dict.items ()),columns = ['column1','column2']) In this short tutorial, you’ll see the complete steps to convert a dictionary to a DataFrame.

How to add dictionary keys and values as pandas columns?

How to Add Dictionary Keys and Values as Pandas Columns? 1 Step 1: Import the necessary libraries#N#Here I will use only the pandas library for creating dataframe.#N#import pandas as... 2 Step 2: Create a List of Dictionary items#N#Before converting a dictionary into the data frame lets creates a sample... 3 Step 3: Create a Dataframe More ...

How do I get a dictionary in Python?

Run the code in Python, and you’ll get the following dictionary: Notice that the syntax of print (type (my_dict)) was add at the bottom of the code to confirm that we indeed got a dictionary.

How to create a Dataframe from a loop?

What you need to do is to build your dictionary with your loop, then at then end of your loop, you can use your dictionary to create a dataframe with: df1 = pd.DataFrame(podcast_dict) And append using pd.concat: df_podcast = pd.concat([df_podcast, df1])


1 Answers

For me below code works:

row = -1
for i in links:
    row = row + 1
    data = urllib2.urlopen(str(i)).read()
    data = json.loads(data)
    for key in data.keys():
        df.loc[row,key] = data[key]

You have mixed order of arguments in .loc() and have one to much []

like image 183
zuku Avatar answered Sep 28 '22 22:09

zuku