Compare list of datetimes with datetime in Python

Question

I have a list of datetime objects and would like to find the ones which are within a certain time frame:

import datetime

dates = [ datetime.datetime(2007, 1, 2, 0, 1),
          datetime.datetime(2007, 1, 3, 0, 2),
          datetime.datetime(2007, 1, 4, 0, 3),
          datetime.datetime(2007, 1, 5, 0, 4),
          datetime.datetime(2007, 1, 6, 0, 5),
          datetime.datetime(2007, 1, 7, 0, 6) ]
#in reality this is a list of over 25000 dates

mask = (dates>datetime.datetime(2007,1,3)) & \
       (dates<datetime.datetime(2007,1,6))

However, this results in the following error: "TypeError: can't compare datetime.datetime to list"

How can I fix my code?

Answer 1

You can mask a numpy.array in the syntax you describe (but not a list):

import numpy as np

date1 = np.array(dates)
mask = (dates1 > datetime.datetime(2007,1,3)) & \
       (dates1 < datetime.datetime(2007,1,6))

In [14]: mask
Out[14]: array([False,  True,  True,  True, False, False], dtype=bool)

In [15]: dates1[mask]
Out[15]: array([2007-01-03 00:02:00, 2007-01-04 00:03:00, 2007-01-05 00:04:00], dtype=object)

(since this question is tagged numpy, presumably this is what you were intending.)

Answer 2

If your dates list is in sorted order, you can use the bisect module:

>>> import bisect
>>> bisect.bisect_right(dates, datetime.datetime(2007,1,3))
1
>>> bisect.bisect_left(dates, datetime.datetime(2007,1,6))
4

The .bisect_* functions return indices into the dates list:

>>> lower = bisect.bisect_right(dates, datetime.datetime(2007,1,3))
>>> upper = bisect.bisect_left(dates, datetime.datetime(2007,1,6))
>>> mask = dates[lower:upper]
>>> mask
[datetime.datetime(2007, 1, 3, 0, 2), datetime.datetime(2007, 1, 4, 0, 3), datetime.datetime(2007, 1, 5, 0, 4)]