Append dictionary to pandas dataframe in a loop

Question

I have a requirement to create a dictionary within a loop and append them to a pandas data frame with matching key name of dict and column name of data frame. The key value pairs of dictionary in each iteration could be different. An empty pandas data frame df_podcast have been defined at the beginning with all possible keys in the dictionary.

Below is the sample of a code which is not completed yet

df_podcast=pd.DataFrame(columns=podcast_cols)

podcast_dict={}
for j in range(len(podcast[0])):
    if podcast[0][j].tag=="key":
        podcast_dict[podcast[0][j].text]=podcast[0][j+1].text
### Have to append dict to pandas df ############

I have append podcast_dict to df_podcast. Podcast is actually a list of lists, here I'm just considering only 1st row of the list

Answer 1

You need:

df  = pd.DataFrame([podcast_dict], columns=podcast_dict.keys())
df_podcast = pd.concat([df_podcast, df], axis =0).reset_index()

Answer 2

If you want to simply append new data from a created dictionary within a loop to an existening Dataframe:

df = pd.DataFrame()
for i in range(n):
    dict_new = dict(i)
    df = df.append(dict_new, ignore_index=True)
print(df)

NOTE: As long as the keys in your created dictionary are the same, appending it to an existing dataframe shouldn't be cumbersome. Source

Answer 3

IIUC:

What you need to do is to build your dictionary with your loop, then at then end of your loop, you can use your dictionary to create a dataframe with:

df1  = pd.DataFrame(podcast_dict)

And append using pd.concat:

df_podcast = pd.concat([df_podcast, df1])