Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas to_datetime() then concat() on DateTime Index

I'm trying to merge 2 DataFrames using concat, on their DateTime Index, but it's not working as I expected. I copied some of this code from the example in the documentation for this example:

import pandas as pd

df = pd.DataFrame({'year': [2015, 2016],
                   'month': [2, 3],
                   'day': [4, 5],
                   'value': [444,555]})

df.set_index(pd.to_datetime(df.loc[:,['year','month','day']]),inplace=True)

df.drop(['year','month','day'],axis=1,inplace=True)

df2 = pd.DataFrame(data=[222,333],
                   index=pd.to_datetime(['2015-02-04','2016-03-05']))

pd.concat([df,df2])
Out[1]: 
            value      0
2015-02-04  444.0    NaN
2016-03-05  555.0    NaN
2015-02-04    NaN  222.0
2016-03-05    NaN  333.0

Why isn't it recognizing the same dates on the index and merging accordingly? I verified that both Indexes are DateTime:

df.index
Out[2]: DatetimeIndex(['2015-02-04', '2016-03-05'], dtype='datetime64[ns]', freq=None)

df2.index
Out[3]: DatetimeIndex(['2015-02-04', '2016-03-05'], dtype='datetime64[ns]', freq=None)

Thanks.

like image 516
Josh D Avatar asked May 10 '17 15:05

Josh D


People also ask

Does PD concat join on index?

pd. concat joins on the index and can join two or more DataFrames at once. It does a full outer join by default. For more information on concat , see this post.

How do I combine date and time columns in pandas?

Pandas Combine() Function combine() function which allows us to take a date and time string values and combine them to a single Pandas timestamp object. The function accepts two main parameters: Date – refers to the datetime. date object denoting the date string.

Is concat faster than append pandas?

Time is of the essence; which one is faster? In this benchmark, concatenating multiple dataframes by using the Pandas. concat function is 50 times faster than using the DataFrame. append version.


1 Answers

pass axis=1 to concatenate column-wise:

In [7]:
pd.concat([df,df2], axis=1)

Out[7]:
            value    0
2015-02-04    444  222
2016-03-05    555  333

Alternatively you could've joined:

In [5]:
df.join(df2)

Out[5]:
            value    0
2015-02-04    444  222
2016-03-05    555  333

or merged:

In [8]:
df.merge(df2, left_index=True, right_index=True)

Out[8]:
            value    0
2015-02-04    444  222
2016-03-05    555  333
like image 56
EdChum Avatar answered Sep 23 '22 09:09

EdChum