Does standard C++11 guarantee that high_resolution_clock measure real time (non CPU-cycles)?

Question

As known clock() may show less than or greater than the value of the real time - both cases are shown in the following examples 1 and 2.

For high-precision measurements of the time in C++11 we can use:

• std::chrono::high_resolution_clock::now(); - guarantee high-precision
• std::chrono::steady_clock::now(); - guarantee that measure real time
• clock(); - guarantee high-precision, but measure CPU-cycles instead of time
• time(&t_start); - isn't high-precision, but measure real time

1- For example: http://ideone.com/SudWTM

#include <stdio.h>
#include <time.h>
#include <thread>
#include <iostream>
#include <chrono>

int main(void) {

    std::cout << "sleep(3) took: \n\n";

    clock_t c_start, c_end;
    time_t t_start, t_end;
    std::chrono::high_resolution_clock::time_point h_start, h_end;
    std::chrono::steady_clock::time_point steady_start, steady_end;

    time(&t_start);  // less precise than clock() but always get the real actual time
    c_start = clock(); // clock() get only CPU-time, it can be more than real or less - sleep(3); took 0.00 seconds 
    h_start = std::chrono::high_resolution_clock::now();
    steady_start = std::chrono::steady_clock::now(); 

    std::this_thread::sleep_for(std::chrono::seconds(3));

    steady_end = std::chrono::steady_clock::now();
    h_end = std::chrono::high_resolution_clock::now();
    c_end = clock();
    time(&t_end);

    std::cout << "highres = " << std::chrono::duration<double>(h_end - h_start).count() << " s \n";
    std::cout << "steady = " << std::chrono::duration<double>(steady_end - steady_start).count() << " s \n";

    printf("clock() = %.2lf seconds \n", (c_end - c_start) / (double)CLOCKS_PER_SEC);
    printf("time() = %.2lf seconds \n", difftime(t_end, t_start));

    return 0;
}

Result on g++ (Debian 4.9.2-10) 4.9.2: clock() = 0.00 seconds

sleep(3) took: 

highres = 3.00098 s 
steady = 3.00098 s 
clock() = 0.00 seconds 
time() = 3.00 seconds 

Result on C++ MSVS 2013 v120 (Windows 7x64):

sleep(3) took:

highres = 3.00017 s
steady = 3.00017 s
clock() = 3.00 seconds
time() = 3.00 seconds

2- Second example OpenMP or <thread>: http://coliru.stacked-crooked.com/a/2922c85385d197e1

#include <stdio.h>
#include <time.h>
#include <thread>
#include <iostream>
#include <chrono>
#include <vector>

int main(void) {

    std::cout << "for-loop took: \n\n";

    clock_t c_start, c_end;
    time_t t_start, t_end;
    std::chrono::high_resolution_clock::time_point h_start, h_end;
    std::chrono::steady_clock::time_point steady_start, steady_end;

    time(&t_start);  // less precise than clock() but always get the real actual time
    c_start = clock(); // clock() get only CPU-time, it can be more than real or less - sleep(3); took 0.00 seconds 
    h_start = std::chrono::high_resolution_clock::now();
    steady_start = std::chrono::steady_clock::now();

    #pragma omp parallel num_threads(10)
    {
        for (volatile int i = 0; i < 200000000; ++i);
    }

    steady_end = std::chrono::steady_clock::now();
    h_end = std::chrono::high_resolution_clock::now();
    c_end = clock();
    time(&t_end);

    std::cout << "highres = " << std::chrono::duration<double>(h_end - h_start).count() << " s \n";
    std::cout << "steady = " << std::chrono::duration<double>(steady_end - steady_start).count() << " s \n";

    printf("clock() = %.2lf seconds \n", (c_end - c_start) / (double)CLOCKS_PER_SEC);
    printf("time() = %.2lf seconds \n", difftime(t_end, t_start));

    int b = getchar();

    return 0;
}

Result on g++ (Debian 4.9.2-10) 4.9.2: clock() = 1.35 seconds

for-loop took: 

highres = 0.213906 s 
steady = 0.213905 s 
clock() = 1.35 seconds 
time() = 0.00 seconds 

Result on C++ MSVS 2013 v120 (Windows 7x64):

for-loop took:

highres = 1.49109 s
steady = 1.49109 s
clock() = 1.49 seconds
time() = 2.00 seconds

Resume:

1. When thread sleeps then clock() on g++ 4.9.2 doesn't measure time unlike other functions.

2. When we use multithreading by using OpenMP or by using <thread> (link), then clock() on g++ 4.9.2 measures CPU-cycles of all threads.

Also on Windows MSVS 2013 clock() measures required real time in both cases, but this doesn't guarantee that clock() measures the same on other platforms (on linux g++ is 0 for the sleep and x-fold for the multithreading).

Based on this, if std::chrono::high_resolution_clock::now(); measures required real time in both cases on both Windows MSVS 2013 and g++ 4.9.2, does this guarantee that it will measure real high resolution time on all other platforms and does whether it guarantee standard C++11/14?

Answer 1

Short answer: as of the C++14 standard, high_resolution_clock does NOT explicitly provide the guarantee you're looking for.

For now, steady_clock and system_clock provide better and more explicit guarantees. However, most implementations probably will ensure that HRC advances while its thread is sleeping. It may nevertheless be preferable to do your own type-aliasing. See 'EDIT' sections below and discussion in comments.

Long answer:

The draft standard does in fact implicitly acknowledge (in note 30.2.4 "Timing Specifications", note 5) that Clock objects are not required to advance while their associated thread is sleeping. For context, this section is explaining how the standard-library timer objects work; the behavior of a timer is based on the behavior of the clock used to set it.

[ Note: If the clock is not synchronized with a steady clock, e.g., a CPU time clock, these timeouts might not provide useful functionality. — end note ]

Note that in this case, "timeouts might not provide useful functionality" means that if you use a timer to sleep_until a particular clock time using an unsynchronized (non-realtime) clock, your thread will not wake up. So the note above is a bit of an understatement.

And, indeed, there is nothing in the Clock specification (20.13.3) that actually requires synchronization with a steady clock.

However, the standard appears to implicitly condone two potential aliases for high_resolution_clock in the definition in 20.13.7.3:

high_resolution_clock may be a synonym for system_clock or steady_clock.

steady_clock is, of course, steady. system_clock is not, because the system time could change (e.g. as the result of an NTP update) while the program is running.

However, system_clock (20.13.7.1) is still a "realtime" clock:

Objects of class system_clock represent wall clock time from the system-wide realtime clock.

So system_clock will not stop advancing when your thread sleeps. This confirms Nicol Bolas's point that a is_steady may be false for high_resolution_clock even if the clock behaves as you expect (i.e. it advances regardless of the state of its associated thread).

Based on this, it seems reasonable to expect most mainstream implementations to use a realtime (i.e. synchronized) clock of some sort for high_resolution_clock. Implementations are designed to be useful, after all, and a clock is generally less useful if it's not realtime, especially if it's used with timers as per the note on "useful functionality" above.

Since it's not guaranteed, however, you should check the behavior and/or documentation of each implementation you want to use.

EDIT: I've started a discussion on the ISO C++ Standards group on the issue, suggesting that this is a bug in the standard. The first reply, from Howard Hinnant, who takes credit for putting it in the standard, is worth quoting:

I would not be opposed to deprecating high_resolution_clock, with the intent to remove it after a suitable period of deprecation. The reality is that it is always a typedef to either steady_clock or system_clock, and the programmer is better off choosing one of those two and know what he’s getting, than choose high_resolution_clock and get some other clock by a roll of the dice.

...So the moral, according to Hinnant, is don't use high_resolution_clock.

EDIT 2:

The problem with high_resolution_clock according to Hinnant is not so much that you're likely to run into a problem with HRC (although that is possible even with a conforming compiler, as per the argument above), but that since you're typically not actually getting a lower resolution than you could with the one of the other two clocks (though you'll need to manually compare their resolutions in a type-alias or typedef to get a "maximum resolution" non-sleeping clock), there's no concrete benefit. So you need to weigh the risk of having threads sleep forever on conforming implementations versus the semantic benefit of the name high_resolution_clock and the simplicity/brevity benefit of avoiding just making your own typedef or type-alias.

Here's some actual code for various approaches:

• Use static_assert to check whether high_resolution_clock is actually aliased to a real clock. This will probably never fire, which means that you're automatically getting the highest-resolution "realtime" clock without messing with your own typedefs:

   static_assert(
        std::is_same<high_resolution_clock, steady_clock>::value
     || std::is_same<high_resolution_clock, system_clock>::value,
     "high_resolution_clock IS NOT aliased to one of the other standard clocks!");
  

• Use the HRC if high_resolution_clock::is_steady is true; otherwise prefer the higher-resolution clock between system_clock and steady_clock. NOTE that if high_resolution_clock::is_steady is false, this probably just means that the HRC is aliased to system_clock, in which case you'll ultimately end up with a new type-alias that is actually the same type as high_resolution_clock. However, creating your own type-alias makes this explicit and guarantees that even a malicious-but-conforming implementation won't have the issue outlined above.

  using maxres_sys_or_steady =
      std::conditional<
          system_clock::period::den <= steady_clock::period::den,
          system_clock, steady_clock
        >::type;
  using maxres_nonsleeping_clock =
      std::conditional<
          high_resolution_clock::is_steady,
          high_resolution_clock, maxres_sys_or_steady
        >::type;
  

Answer 2

The standard does not specify this behavior from its clocks. Not exactly.

A clock has the is_steady static property, which can be checked. Any clock for which is_steady returns true cannot be the sort of clock that stops running just because you put a thread to sleep. However, a clock for which that value is false could be non-steady for a variety of reasons. It may not be steady because it's a wall clock that will change if the system time changes. Or because the period between ticks is an average, rather than an exact number.

So is_steady does not really answer your question.

The standard does not specify high_resolution_clock::is_steady, but it does require the implementation to answer that question. If it is steady, then you are guaranteed that sleeping a thread won't stop the clock. But if it's not steady... you get no guarantee at all.