I have a set of urls containing json files and an empty pandas dataframe with columns representing the attributes of the jsnon files. Not all json files have all the attributes in the pandas dataframe. What I need to do is to create dictionaries out of the json files and then append each dictionary to the pandas dataframe as a new row and, in case the json file doesn't have an attribute matching a column in the dataframe this has to be filled blank.
I managed to create dictionaries as:
import urllib2
import json
url = "https://cws01.worldstores.co.uk/api/product.php?product_sku=ULST:7BIS01CF"
data = urllib2.urlopen(url).read()
data = json.loads(data)
and then I tried to create a for loop as follows:
row = -1
for i in links:
row = row + 1
data = urllib2.urlopen(str(i)).read()
data = json.loads(data)
for key in data.keys():
for column in df.columns:
if str(column) == str(key):
df.loc[[str(column)],row] = data[str(key)]
else:
df.loc[[str(column)],row] = None
where df is the dataframe and links is the set of urls
However, I get the following error:
raise KeyError('%s not in index' % objarr[mask])
KeyError: "['2_seater_depth_mm'] not in index"
where ['2_seater_depth_mm'] is the first column of the pandas dataframe
You may use the following template to convert a dictionary to Pandas DataFrame: import pandas as pd my_dict = {key:value,key:value,key:value,...} df = pd.DataFrame (list (my_dict.items ()),columns = ['column1','column2']) In this short tutorial, you’ll see the complete steps to convert a dictionary to a DataFrame.
How to Add Dictionary Keys and Values as Pandas Columns? 1 Step 1: Import the necessary libraries#N#Here I will use only the pandas library for creating dataframe.#N#import pandas as... 2 Step 2: Create a List of Dictionary items#N#Before converting a dictionary into the data frame lets creates a sample... 3 Step 3: Create a Dataframe More ...
Run the code in Python, and you’ll get the following dictionary: Notice that the syntax of print (type (my_dict)) was add at the bottom of the code to confirm that we indeed got a dictionary.
What you need to do is to build your dictionary with your loop, then at then end of your loop, you can use your dictionary to create a dataframe with: df1 = pd.DataFrame(podcast_dict) And append using pd.concat: df_podcast = pd.concat([df_podcast, df1])
For me below code works:
row = -1
for i in links:
row = row + 1
data = urllib2.urlopen(str(i)).read()
data = json.loads(data)
for key in data.keys():
df.loc[row,key] = data[key]
You have mixed order of arguments in .loc()
and have one to much []
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With