I have Three dataframes. All of them have a common column and I need to merge them based on the common column without missing any data
Input
>>>df1 0 Col1 Col2 Col3 1 data1 3 4 2 data2 4 3 3 data3 2 3 4 data4 2 4 5 data5 1 4 >>>df2 0 Col1 Col4 Col5 1 data1 7 4 2 data2 6 9 3 data3 1 4 >>>df3 0 Col1 Col6 Col7 1 data2 5 8 2 data3 2 7 3 data5 5 3
Expected Output
>>>df 0 Col1 Col2 Col3 Col4 Col5 Col6 Col7 1 data1 3 4 7 4 2 data2 4 3 6 9 5 8 3 data3 2 3 1 4 2 7 4 data4 2 4 5 data5 1 4 5 3
To merge two Pandas DataFrame with common column, use the merge() function and set the ON parameter as the column name.
To join these DataFrames, pandas provides multiple functions like concat() , merge() , join() , etc. In this section, you will practice using merge() function of pandas. You can notice that the DataFrames are now merged into a single DataFrame based on the common values present in the id column of both the DataFrames.
Use merge
and reduce
In [86]: from functools import reduce
In [87]: reduce(lambda x,y: pd.merge(x,y, on='Col1', how='outer'), [df1, df2, df3])
Out[87]:
Col1 Col2 Col3 Col4 Col5 Col6 Col7
0 data1 3 4 7.0 4.0 NaN NaN
1 data2 4 3 6.0 9.0 5.0 8.0
2 data3 2 3 1.0 4.0 2.0 7.0
3 data4 2 4 NaN NaN NaN NaN
4 data5 1 4 NaN NaN 5.0 3.0
Details
In [88]: df1
Out[88]:
Col1 Col2 Col3
0 data1 3 4
1 data2 4 3
2 data3 2 3
3 data4 2 4
4 data5 1 4
In [89]: df2
Out[89]:
Col1 Col4 Col5
0 data1 7 4
1 data2 6 9
2 data3 1 4
In [90]: df3
Out[90]:
Col1 Col6 Col7
0 data2 5 8
1 data3 2 7
2 data5 5 3
Using pd.concat
:
df1.set_index('Col1',inplace=True)
df2.set_index('Col1',inplace=True)
df3.set_index('Col1',inplace=True)
df = pd.concat([df1,df2,df3],axis=1,sort=False).reset_index()
df.rename(columns = {'index':'Col1'})
Col1 Col2 Col3 Col4 Col5 Col6 Col7
0 data1 3 4 7.0 4.0 NaN NaN
1 data2 4 3 6.0 9.0 5.0 8.0
2 data3 2 3 1.0 4.0 2.0 7.0
3 data4 2 4 NaN NaN NaN NaN
4 data5 1 4 NaN NaN 5.0 3.0
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With