Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Subtracting pandas timestamps; absolute value

Tags:

python

pandas

I have been playing a bit to try to understand pandas timestamps and timedeltas. I like how you can operate with them, but when trying subtraction I found this a bit odd:

now = pd.Timestamp('now')
then = now - pd.to_timedelta('1h')

print (now - then)
print (then - now)
print ((now - then).seconds)
print ((then - now).seconds)

Results in:

0 days 01:00:00
-1 days +23:00:00
3600
82800

a) How should I understand this behavior?

b) Is there a way to have an absolute value of the difference of timestamps, the equivalent to abs()?

like image 478
luna1999 Avatar asked Aug 05 '15 15:08

luna1999


1 Answers

The reason for this seemingly strange/buggy behaviour is that the .seconds attribute of a timedelta (for pandas.Timedelta, but this is inherited from standard library's timedelta.timedelta) is very ambigous.
The timedelta is stored in 3 parts: days, seconds, microseconds (https://docs.python.org/2/library/datetime.html#timedelta-objects). So the seconds is the sum of hours, minutes and seconds (in seconds).

So there are 2 'strange' things that can lead to confusion:

  • When having a negative timedelta, you get -1 days +23:00:00 instead of -01:00:00. This is because only the days part can be negative. So a negative timedelta will always be defined as a negative number of days with adding hours or seconds again to get the correct value. So this gives you the +23h part.
  • The seconds is the sum of hours, minutes and seconds. So the +23:00:00 we get is equal to 82800 seconds.

Bottomline is, the .seconds attribute of a timedelta does not give you the seconds part neither the total seconds (timedelta converted to seconds). So in practice, I think you should almost never use it.

To obtain the timedelta in seconds, you can use the total_seconds method. If I define the negative difference to diff = then - now:

In [12]: diff
Out[12]: Timedelta('-1 days +23:00:00')

In [13]: diff.seconds
Out[13]: 82800

In [14]: diff.total_seconds()
Out[14]: -3600.0
like image 172
joris Avatar answered Oct 15 '22 14:10

joris