Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python pandas: How to slice dataframe using 13 digit timestamp

Tags:

python

pandas

I have the following dataframe:

              DateTime                   Seq
timestamp
1475504294990,10/03/2016 10:18:14:990000,2123847
1475504446660,10/03/2016 10:20:46:660000,2123908
1475504524410,10/03/2016 10:22:04:410000,2123953
1475504848100,10/03/2016 10:27:28:100000,2124067
1475504940530,10/03/2016 10:29:00:530000,2124126

i want to slice this dataframe using a start and end time stamp

start = 1475504446660
end = 1475504848100
print df[start:end]
              DateTime                   Seq
timestamp
1475504446660,10/03/2016 10:20:46:660000,2123908
1475504524410,10/03/2016 10:22:04:410000,2123953
1475504848100,10/03/2016 10:27:28:100000,2124067

However,I am getting this error:

IndexError: failed to coerce slice entry of type long to integer

I tried using df[int(start):int(end)], still getting same error


1 Answers

To slice you have to define the timestamp as the index and use loc to perform label indexing (else it is ambiguous between position and label indexing for integer indexes).

df = df.set_index('timestamp')
df.loc[start:end]

#                                  DateTime      Seq
# timestamp                                         
# 1475504446660  10/03/2016 10:20:46:660000  2123908
# 1475504524410  10/03/2016 10:22:04:410000  2123953
# 1475504848100  10/03/2016 10:27:28:100000  2124067

By default in the case of an integer index the indexing is made by position and not by label, see the result in this example.

df[0:2] # equivalent to df.iloc[0:2]

#                                  DateTime      Seq
# timestamp                                         
# 1475504294990  10/03/2016 10:18:14:990000  2123847
# 1475504446660  10/03/2016 10:20:46:660000  2123908

Note

If you do not want to define timestamp as the index you can use this syntax to obtain the same result.

df.query('@start <= timestamp <= @end')

#        timestamp                    DateTime      Seq
# 1  1475504446660  10/03/2016 10:20:46:660000  2123908
# 2  1475504524410  10/03/2016 10:22:04:410000  2123953
# 3  1475504848100  10/03/2016 10:27:28:100000  2124067
like image 108
Romain Avatar answered Jul 17 '26 22:07

Romain