Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Merge dataframes in a dictionary

Tags:

python

pandas

Say I have an dictionary of dataframes:

  {'df1': name         color    type           Apple        Yellow   Fruit,    'df2': name         color    type           Banana       Red      Fruit,    'df3': name         color    type           Chocolate    Brown    Sweet     ......} 

And I want to merge them all into one like this:

  name         color    type   Apple        Red      Fruit    Banana       Yellow   Fruit   Chocolate    Brown    Sweet 

I can do it manually as follows:

  merge1=pd.merge('df1','df2')   merge2=pd.merge('merge1','df3')   ... 

But is there a way to automatically zip through the dictionary and merge? Any help is appreciated.

like image 814
qts Avatar asked May 14 '15 09:05

qts


People also ask

How do you concatenate a dictionary to a DataFrame in Python?

concat() function If a dict is passed, the sorted keys will be used as the keys argument, unless it is passed, in which case the values will be selected (see below). The axis to concatenate along. Handle indexes on other axis (or axes). Field name to join on in left DataFrame.

Can you have a dictionary of DataFrames?

A dataframe is a data structure constructed with rows and columns, similar to a database or Excel spreadsheet. It consists of a dictionary of lists in which the list each have their own identifiers or keys, such as “last name” or “food group.”

How do I merge two DataFrames in Python?

The concat() function can be used to concatenate two Dataframes by adding the rows of one to the other. The merge() function is equivalent to the SQL JOIN clause. 'left', 'right' and 'inner' joins are all possible.

How do you convert a DataFrame to a dictionary?

We can convert a dictionary to a pandas dataframe by using the pd. DataFrame. from_dict() class-method.


1 Answers

You can just pass the dict direct and access the values attribute to concat:

In [233]:  d Out[233]: {'df1':     name         color    type  0  Apple        Yellow   Fruit, 'df2':     name         color    type  0  Banana       Red      Fruit, 'df3':     name         color    type  0  Chocolate    Brown    Sweet} In [234]:  pd.concat(d.values(), ignore_index=True) Out[234]:     name         color    type 0  Banana       Red      Fruit 1  Apple        Yellow   Fruit 2  Chocolate    Brown    Sweet 

This assumes that you are just looking to concatenate all the dfs, if you are going to merge then you need to explain what the merge criteria is

like image 166
EdChum Avatar answered Sep 30 '22 06:09

EdChum