Measuring elapsed time in python

Question

Is there a simple way / module to correctly measure the elapsed time in python? I know that I can simply call time.time() twice and take the difference, but that will yield wrong results if the system time is changed. Granted, that doesn't happen very often, but it does indicate that I'm measuring the wrong thing.

Using time.time() to measure durations is incredibly roundabout when you think about it. You take the difference of two absolute time measurements which are in turn constructed from duration measurements (performed by timers) and known absolute times (set manually or via ntp), that you aren't interested in at all.

So, is there a way to query this "timer time" directly? I'd imagine that it can be represented as a millisecond or microsecond value that has no meaningful absolute representation (and thus doesn't need to be adjusted with system time). Looking around a bit it seems that this is exactly what System.nanoTime() does in Java, but I did not find a corresponding Python function, even though it should (hardware-technically) be easier to provide than time.time().

Edit: To avoid confusion and address the answers below: This is not about DST changes, and I don't want CPU time either - I want elapsed physical time. It doesn't need to be very fine-grained, and not even particularly accurate. It just shouldn't give me negative durations, or durations which are off by several orders of magnitude (above the granularity), just because someone decided to set the system clock to a different value. Here's what the Python docs say about 'time.time()':

"While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between the two calls"

This is exactly what I want to avoid, since it can lead to strange things like negative values in time calculations. I can work around this at the moment, but I believe it is a good idea to learn using the proper solutions where feasible, since the kludges will come back to bite you one day.

Edit2: Some research shows that you can get a system time independent measurement like I want in Windows by using GetTickCount64(), under Linux you can get it in the return value of times(). However, I still can't find a module which provides this functionality in Python.

Answer 1

For measuring elapsed CPU time, look at time.clock(). This is the equivalent of Linux's times() user time field.

For benchmarking, use timeit.

The datetime module, which is part of Python 2.3+, also has microsecond time if supported by the platform.

Example:

>>> import datetime as dt >>> n1=dt.datetime.now() >>> n2=dt.datetime.now() >>> (n2-n1).microseconds 678521 >>> (n2.microsecond-n1.microsecond)/1e6 0.678521 ie, it took me .678521 seconds to type the second n2= line -- slow >>> n1.resolution datetime.timedelta(0, 0, 1) 1/1e6 resolution is claimed. 

If you are concerned about system time changes (from DS -> ST) just check the object returned by datetime.Presumably, the system time could have a small adjustment from an NTP reference adjustment. This should be slewed, and corrections are applied gradually, but ntp sync beats can have an effect with very small (millisec or microsec) time references.

You can also reference Alex Martelli's C function if you want something of that resolution. I would not go too far to reinvent the wheel. Accurate time is basic and most modern OS's do a pretty good job.

Edit

Based on your clarifications, it sounds like you need a simple side check if the system's clock has changed. Just compare to a friendly, local ntp server:

import socket import struct import time  ntp="pool.ntp.org"   # or whatever ntp server you have handy  client = socket.socket( socket.AF_INET, socket.SOCK_DGRAM ) data = '\x1b' + 47 * '\0' client.sendto( data, ( ntp, 123 )) data, address = client.recvfrom( 1024 ) if data:     print 'Response received from:', address     t = struct.unpack( '!12I', data )[10]     t -= 2208988800L #seconds since Epoch     print '\tTime=%s' % time.ctime(t) 

NTP is accurate to milliseconds over the Internet and has representation resolution of resolution of 2−32 seconds (233 picoseconds). Should be good enough?

Be aware that the NTP 64 bit data structure will overflow in 2036 and every 136 years thereafter -- if you really want a robust solution, better check for overflow...

Answer 2

What you seem to be looking for is a monotonic timer. A monotonic time reference does not jump or go backwards.

There have been several attempts to implement a cross platform monotomic clock for Python based on the OS reference of it. (Windows, POSIX and BSD are quite different) See the discussions and some of the attempts at monotonic time in this SO post.

Mostly, you can just use os.times():

os.times()

Return a 5-tuple of floating point numbers indicating accumulated (processor or other) times, in seconds. The items are: user time, system time, children’s user time, children’s system time, and elapsed real time since a fixed point in the past, in that order. See the Unix manual page times(2) or the corresponding Windows Platform API documentation. On Windows, only the first two items are filled, the others are zero.

Availability: Unix, Windows

But that does not fill in the needed elapsed real time (the fifth tuple) on Windows.

If you need Windows support, consider ctypes and you can call GetTickCount64() directly, as has been done in this recipe.