Clock synchronization quality on Windows Azure?

Question

I am looking for quantitative estimates on clock offsets between VMs on Windows Azure - assuming that all VMs are hosted in the same datacenter. I am guesstimating that average clock offset between one VM and another is below 10 seconds, but I am not even sure it's guaranteed property of the Azure cloud.

Has anybody some quantitative measurements on that matter?

Answer 1

I have finally settled to do some experiments on my own.

A few facts concerning the experiment protocol:

• Instead of looking for offset to an reference clock, I have simply checked clock differences between Azure VMs and the Azure Storage.
• Clock time of the Azure Storage has been retrieved using the HTTP hack pasted below.
• Measurements have been done within the North Europe datacenter of Azure with 250 small VMs.
• Latency between storage and VMs measured with Stopwatch was always lower than 1ms for minimalistic unauthenticated requests (basically HTTP requests were coming back with 400 errors, but still with Date: available in the HTTP headers).

Results:

• About 50% of the VMs have a clock offset to the storage greater than 1s.
• About 5% of the VMs have a clock offset to the storage greater than 2s.
• Less than 1% observations for clock offsets close 3s.
• A handfew outliers close to 4s.
• The clock offset between a single VM and the storage typically vary of +1/-1s from one request to the next.

So technically, we are not too far from the 2s tolerance target, although for intra-data-center sync, you don't have to push the experiment far to observe close to 4s offset. If we assume a normal (aka Gaussian) distribution for the clock offsets, then I would say that relying on any clock threshold lower than 6s is bound to lead to scheduling issues.

/// <summary> /// Substitute for proper NTP (Network Time Protocol)  /// when UDP is not available, as on Windows Azure. /// </summary> public class HttpTimeChecker {     public static DateTime GetUtcNetworkTime(string server)     {         // HACK: we can't use WebClient here, because we get a faulty HTTP response         // We don't care about HTTP error, the only thing that matter is the presence         // of the 'Date:' HTTP header         var tc = new TcpClient();         tc.Connect(server, 80);          string response;         using (var ns = tc.GetStream())         {             var sw = new StreamWriter(ns);             var sr = new StreamReader(ns);              string req = "";             req += "GET / HTTP/1.0\n";             req += "Host: " + server + "\n";             req += "\n";              sw.Write(req);             sw.Flush();              response = sr.ReadToEnd();         }          foreach(var line in response.Split(new[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries))         {             if(line.StartsWith("Date: "))             {                 return DateTime.Parse(line.Substring(6)).ToUniversalTime();             }         }          throw new ArgumentException("No date to be retrieved among HTTP headers.", "server");     } }