Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Select only rows that occur at specific time

I have read in C.csv and the datetime column is a object type.

I want to get every row that has 23:45:00 in it, regardless of date. I would like to have datetime as index and i would like to convert datetime index to datetime64[ns]. I believe pandas is designed for this sort of thing but I'm getting my indexes and data-types mixed up.

    import datetime as dt
    import pandas as pd
    df = pd.read_csv('C.csv', index_col = 'datetime', parse_dates=['datetime'])

The dataframe:

                     C      H      L      O  OI  V    WAP
datetime                                                     
2017-04-22 09:23:00  39.48  39.48  39.48  39.48   0  0  39.48
2017-04-22 09:24:00  39.48  39.48  39.48  39.48   0  0  39.48
2017-04-22 09:25:00  39.48  39.48  39.48  39.48   0  0  39.48
2017-04-22 09:26:00  39.44  39.44  39.44  39.44   1  4  39.44
2017-04-22 09:27:00  39.48  39.48  39.48  39.48   3  2  39.48
like image 793
Ross Demtschyna Avatar asked May 14 '17 23:05

Ross Demtschyna


1 Answers

print(df)
              datetime      C      H      L      O  OI  V    WAP
0  2017-04-22 09:23:00  39.48  39.48  39.48  39.48   0  0  39.48
1  2017-04-22 09:24:00  39.48  39.48  39.48  39.48   0  0  39.48
2  2017-04-22 09:25:00  39.48  39.48  39.48  39.48   0  0  39.48
3  2017-04-22 09:26:00  39.44  39.44  39.44  39.44   1  4  39.44
4  2017-04-22 09:27:00  39.48  39.48  39.48  39.48   3  2  39.48
5  2017-04-23 09:25:00  39.48  39.48  39.48  39.48   3  2  39.48

Make datetime an index, and convert to datetime dtype:

df.set_index('datetime', inplace=True)
df.index = pd.to_datetime(df.index)

print(df.index.dtype)
dtype('<M8[ns]')

Now set matching timestamp to desired time and filter by matches:

match_timestamp = "09:25:00"
df.loc[df.index.strftime("%H:%M:%S") == match_timestamp]

                         C      H      L      O  OI  V    WAP
datetime                                                     
2017-04-22 09:25:00  39.48  39.48  39.48  39.48   0  0  39.48
2017-04-23 09:25:00  39.48  39.48  39.48  39.48   3  2  39.48

(The timestamp 23:45:00 was not included in your example data, but to match on this time instead, just adjust match_timestamp.)

like image 130
andrew_reece Avatar answered Oct 01 '22 15:10

andrew_reece