Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: How to turn a dictionary of Dataframes into one big dataframe with column names being the key of the previous dict?

So my dataframe is made from lots of individual excel files, each with the the date as their file name and the prices of the fruits on that day in the spreadsheet, so the spreadsheets look something like this:

15012016: Fruit     Price Orange    1 Apple     2 Pear      3  16012016: Fruit     Price Orange    4 Apple     5 Pear      6  17012016: Fruit     Price Orange    7 Apple     8 Pear      9 

So to put all that information together I run the following code to put all the information into a dictionary of dataframes (all fruit price files stored in 'C:\Fruit_Prices_by_Day'

#find all the file names file_list = [] for x in os.listdir('C:\Fruit_Prices_by_Day'):     file_list.append(x)   file_list= list(set(file_list))  d = {}  for date in Raw_list:     df1 = pd.read_excel(os.path.join('C:\Fruit_Prices_by_Day', date +'.xlsx'), index_col = 'Fruit')     d[date] = df1 

Then this is the part where I'm stuck. How do I then make this dict into a dataframe where the column names are the dict keys i.e. the dates, so I can get the price of each fruit per day all in the same dataframe like:

          15012016   16012016   17012016    Orange    1          4          7 Apple     2          5          8 Pear      3          6          9 
like image 587
pakkunrob Avatar asked Mar 01 '16 08:03

pakkunrob


People also ask

How do I convert a dictionary to a DataFrame in Python?

You can convert a dictionary to Pandas Dataframe using df = pd. DataFrame. from_dict(my_dict) statement.

Can I convert dictionary to DataFrame?

We can convert a dictionary to a pandas dataframe by using the pd. DataFrame. from_dict() class-method.

How do I combine multiple Dataframes into one in Python?

The concat() function can be used to concatenate two Dataframes by adding the rows of one to the other. The merge() function is equivalent to the SQL JOIN clause. 'left', 'right' and 'inner' joins are all possible.

How do I merge two Dataframes in Python with different column names?

Different column names are specified for merges in Pandas using the “left_on” and “right_on” parameters, instead of using only the “on” parameter. Merging dataframes with different names for the joining variable is achieved using the left_on and right_on arguments to the pandas merge function.


1 Answers

You can try first set_index of all dataframes in comprehension and then use concat with remove last level of multiindex in columns:

 print d {'17012016':     Fruit  Price 0  Orange      7 1   Apple      8 2    Pear      9, '16012016':     Fruit  Price 0  Orange      4 1   Apple      5 2    Pear      6, '15012016':     Fruit  Price 0  Orange      1 1   Apple      2 2    Pear      3} d = { k: v.set_index('Fruit') for k, v in d.items()}  df = pd.concat(d, axis=1) df.columns = df.columns.droplevel(-1)  print df         15012016  16012016  17012016 Fruit                                Orange         1         4         7 Apple          2         5         8 Pear           3         6         9 
like image 189
jezrael Avatar answered Oct 08 '22 18:10

jezrael