Why does Sleep() slow down subsequent code for 40ms?

Question

I originally asked about this at coderanch.com, so if you've tried to assist me there, thanks, and don't feel obliged to repeat the effort. coderanch.com is mostly a Java community, though, and this appears (after some research) to really be a Windows question, so my colleagues there and I thought this might be a more appropriate place to look for help.

I have written a short program that either spins on the Windows performance counter until 33ms have passed, or else calls Sleep(33). The former exhibits no unexpected effects, but the latter appears to (inconsistently) slow subsequent processing for about 40ms (either that, or it has some effect on the values returned from the performance counter for that long). After the spin or Sleep(), the program calls a routine, runInPlace(), that spins for 2ms, counting the number of times it queries the performance counter, and returning that number.

When the initial 33ms delay is done by spinning, the number of iterations of runInPlace() tends to be (on my Windows 10, XPS-8700) about 250,000. It varies, probably due to other system overhead, but it varies smoothing around 250,000.

Now, when the initial delay is done by calling Sleep(), something strange happens. A lot of the calls to runInPlace() return a number near 250,000, but quite a few of them return a number near 50,000. Again, the range varies around 50,000, fairly smoothly. But, it is clearly averaging one or the other, with nearly no returns anywhere between 80,000 and 150,000. If I call runInPlace() 100 times after each delay, instead of just once, it never returns a number of iterations in the smaller range after the 20th call. As runInPlace() runs for 2ms, this means the behavior I'm observing disappears after 40ms. If I have runInPlace() run for 4ms instead of 2ms, it never returns a number of iterations in the smaller range after the 10th call, so, again, the behavior disappears after 40ms (likewise if have runInPlace() run for only 1ms; the behavior disappears after the 40th call).

Here's my code:

#include "stdafx.h"
#include "Windows.h"

int runInPlace(int msDelay)
{
    LARGE_INTEGER t0, t1;
    int n = 0;

    QueryPerformanceCounter(&t0);

    do
    {
            QueryPerformanceCounter(&t1);
            n++;
    } while (t1.QuadPart - t0.QuadPart < msDelay);

    return n;
}

int _tmain(int argc, _TCHAR* argv[])
{
    LARGE_INTEGER t0, t1;
    LARGE_INTEGER frequency;
    int n;

    QueryPerformanceFrequency(&frequency);

    int msDelay = 2 * frequency.QuadPart / 1000;

    int spinDelay = 33 * frequency.QuadPart / 1000;

    for (int i = 0; i < 100; i++)
    {
        if (argc > 1)
            Sleep(33);
        else
        {
            QueryPerformanceCounter(&t0);

            do
            {
                    QueryPerformanceCounter(&t1);
            } while (t1.QuadPart - t0.QuadPart < spinDelay);
        }

        n = runInPlace(msDelay);
        printf("%d \n", n);
    }

    getchar();

    return 0;
}

Here's some output typical of what I get when using Sleep() for the delay:

56116 248936 53659 34311 233488 54921 47904 45765 31454 55633 55870 55607 32363 219810 211400 216358 274039 244635 152282 151779 43057 37442 251658 53813 56237 259858 252275 251099

And here's some output typical of what I get when I spin to create the delay:

276461 280869 276215 280850 188066 280666 281139 280904 277886 279250 244671 240599 279697 280844 159246 271938 263632 260892 238902 255570 265652 274005 273604 150640 279153 281146 280845 248277

Can anyone help me understand this behavior? (Note, I have tried this program, compiled with Visual C++ 2010 Express, on five computers. It only shows this behavior on the two fastest machines I have.)

Answer 1

This sounds like it is due to the reduced clock speed that the CPU will run at when the computer is not busy (SpeedStep). When the computer is idle (like in a sleep) the clock speed will drop to reduce power consumption. On newer CPUs this can be 35% or less of the listed clock speed. Once the computer gets busy again there is a small delay before the CPU will speed up again.

You can turn off this feature (either in the BIOS or by changing the "Minimum processor state" setting under "Processor power management" in the advanced settings of your power plan to 100%.

Answer 2

Besides what @1201ProgramAlarm said (which may very well be, modern processors are extremely fond of downclocking whenever they can), it may also be a cache warming up problem.

When you ask to sleep for a while the scheduler typically schedules another thread/process for the next CPU time quantum, which means that the caches (instruction cache, data cache, TLB, branch predictor data, ...) relative to your process are going to be "cold" again when your code regains the CPU.