Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Slice efficiently pandas datetime index by a specific time

Tags:

python

pandas

What is the most efficient way to get all the rows of a dataframe with a specific time? For instance, if I create the following DataFrame,

df = DataFrame(index=pd.date_range('2010-01-01', '2016-04-01',freq='min'))

and then try to get all the rows with a 3pm time:

%timeit df[df.index.time == time(15,0)]
1 loops, best of 3: 9.29 s per loop

it works but it is very slow.

Also, what about efficiently slicing between two specific times ?

%timeit df[(df.index.time >= time(15,0)) & (df.index.time <= time(16,0))]
1 loops, best of 3: 18.7 s per loop
like image 213
graham Avatar asked Apr 24 '16 10:04

graham


People also ask

What does Datetimeindex do in pandas?

Immutable ndarray of datetime64 data, represented internally as int64, and which can be boxed to Timestamp objects that are subclasses of datetime and carry metadata such as frequency information. If data is None, start is used as the start point in generating regular timestamp data.

How do I move the datetime index in pandas?

shift() function shift index by desired number of time frequency increments. This method is for shifting the values of datetime-like indexes by a specified time increment a given number of times.

How do pandas deal with date time?

Pandas has a built-in function called to_datetime()that converts date and time in string format to a DateTime object. As you can see, the 'date' column in the DataFrame is currently of a string-type object. Thus, to_datetime() converts the column to a series of the appropriate datetime64 dtype.


2 Answers

You can use at_time and between_time:

print df.at_time('15:00')

print df.between_time(start_time='15:00', end_time='16:00')
like image 171
jezrael Avatar answered Nov 14 '22 21:11

jezrael


try this:

df.loc[df.index.hour == 15]
like image 40
MaxU - stop WAR against UA Avatar answered Nov 14 '22 23:11

MaxU - stop WAR against UA