Determining program runtimes on core i5/7 architecture

Question

When determining accurately how fast an algorithm runs in my programs, I've always used QueryPerformanceCounter() in conjunction with QueryPerformanceFrequency(), however what happens when I'm using a core i5/7 architecture?

Is it possible that turbo boost would suddenly kick in half way through my algorithm and suddenly my performance timer is no longer accurate because the frequency of my clock is no longer a constant or is this really not an issue? If it is an issue, what would be a better way to accurately time an algorithm?

Answer 1

To put it simply, Yes, anything can happen. Modern CPU's just plain make estimating real world performance to damn hard to do. This is not limited to just the latest Intel x86-64 cpus, it applies equally to 8bit PIC microcontrollers, and everything inbetween.

The only reasonable thing to do about it is to just test more. You will need to perform many repeated tests to get an accurate idea about real run-time performance on real hardware for real workloads.

On the other hand, if one algorithm outperforms another on the same hardware ("on My machine") then it will usually give similar comparative results on other setups, though it may vary by a constant factor.

Answer 2

Here's an interesting article on the subject of QueryPerformanceCounter:

http://www.virtualdub.org/blog/pivot/entry.php?id=106

Apparently, the VirtualDub dude stopped using it, and now favors timeGetTime. A good read.

I recommend using Boost.Date_Time, specifically the boost::posix_time stuff.

Answer 3

There are actually two separate issues here:

1. reliability of any given timing method when CPU clock frequency changes (either increasing via e.g. Turbo Boost or decreasing because of e.g. power saving or thermal management kicking in) - this is addressed in some of the other answers

2. reliability of a benchmark measurement (given a reliable clock) - this is more problematic - if you're working on optimisation and looking for small improvements, e.g. around 10%, then it's easy to get misled by the effect of clock variations from Turbo Boost at al. Possible solutions are:

   2.1. disable Turbo Boost and other clock frequency scaling

   2.2. run benchmarks for a long enough time to average out clock speed variations