rdtsc accuracy across CPU cores

Question

I am sending network packets from one thread and receiving replies on a 2nd thread that runs on a different CPU core. My process measures the time between send & receive of each packet (similar to ping). I am using rdtsc for getting high-resolution, low-overhead timing, which is needed by my implementation.

All measurments looks reliable. Still, I am worried about rdtsc accuracy across cores, since I've been reading some texts which implied that tsc is not synced between cores.

I found the following info about TSC in wikipedia

Constant TSC behavior ensures that the duration of each clock tick is uniform and supports the use of the TSC as a wall clock timer even if the processor core changes frequency. This is the architectural behavior moving forward for all Intel processors.

Still I am worried about accruracy across cores, and this is my question

More Info

• I run my process on an Intel nehalem machine.
• Operating System is Linux.
• The "constant_tsc" cpu flag is set for all the cores.

Answer 1

X86_FEATURE_CONSTANT_TSC + X86_FEATURE_NONSTOP_TSC bits in cpuid (edx=x80000007, bit #8; check unsynchronized_tsc function of linux kernel for more checks)

Intel's Designer's vol3b, section 16.11.1 Invariant TSC it says the following

"16.11.1 Invariant TSC

The time stamp counter in newer processors may support an enhancement, referred to as invariant TSC. Processor's support for invariant TSC is indicated by CPUID.80000007H:EDX[8].

The invariant TSC will run at a constant rate in all ACPI P-, C-. and T-states. This is the architectural behavior moving forward. On processors with invariant TSC support, the OS may use the TSC for wall clock timer services (instead of ACPI or HPET timers). TSC reads are much more efficient and do not incur the overhead associated with a ring transition or access to a platform resource."

So, if TSC can be used for wallclock, they are guaranteed to be in sync.

Answer 2

In fact, it seems that cores doesn´t share TSC, check this thread: http://software.intel.com/en-us/forums/topic/388964

Summarizing, different cores does not share TSC, sometimes TSC can get out of synchronization if a core change to an specific energy state, but it depends on the kind of CPU, so you need to check the Intel documentation. It seems that most Operating Systems synchronize TSC on boot.
I checked the differences between TSC on different cores, using an exciting-reacting algorithm, on a Linux Debian machine with core i5 processor. The exciter process (in one core) writed the TSC in a shared variable, when the reacting process detected a change in that variable it compares its value and compares it with its own TSC. This is an example output of my test program:

TSC ping-pong test result: TSC cores (exciter-reactor): 0-1 100 records, avrg: 159, range: 105-269 Dispersion: 13 TSC ping-pong test result: TSC cores (exciter-reactor): 1-0 100 records, avrg: 167, range: 125-410 Dispersion: 13 

The reaction time when the exciter CPU is 0 (159 tics on average) is almost the same than when the exciter CPU is 1 (167 tics). This indicates that they are pretty well synchronized (perhaps with a few tics of difference). On other core pairs, results were very similar.
On the other hand, rdtscp assembly instruction return a value indicating the CPU in which the TSC was read. It is not your case but it can be useful when you want to measure time in a simple code segment and you want to ensure that the process was not moved of CPU in the middle of the code.