Joining Table/DataFrames with common Column in Python

Question

I have two DataFrames:

df1 = ['Date_Time',
    'Temp_1',
    'Latitude',
    'N_S',
    'Longitude',
    'E_W']

df2 = ['Date_Time',
    'Year',
    'Month',
    'Day',
    'Hour',
    'Minute',
    'Seconds']

As You can see both DataFrames have Date_Time as a common column. I want to Join these two DataFrames by matching Date_Time.

My current code is: df.join(df2, on='Date_Time'), but this is giving an error.

Answer 1

You are looking for a merge:

df1.merge(df2, on='Date_Time')

The keywords are the same as for join, but join uses only the index, see "Database-style DataFrame joining/merging".

Here's a simple example:

import pandas as pd
df1 = pd.DataFrame([[1, 2, 3]])
df2 = pd.DataFrame([[1, 7, 8],[4, 9, 9]], columns=[0, 3, 4])

In [4]: df1
Out[4]: 
   0  1  2
0  1  2  3

In [5]: df2
Out[5]: 
   0  3  4
0  1  7  8
1  4  9  9

In [6]: df1.merge(df2, on=0)
Out[6]: 
   0  1  2  3  4
0  1  2  3  7  8

In [7]: df1.merge(df2, on=0, how='outer')
Out[7]: 
   0   1   2  3  4
0  1   2   3  7  8
1  4 NaN NaN  9  9

If you try and join on a column you get an error:

In [8]: df1.join(df2, on=0)
# error!
Exception: columns overlap: array([0], dtype=int64)

See "Joining key columns on an index".