Protocol Buffers: Should I use int64 or fixed64 to represent a .NET DateTime value?

Question

I need to serialize a .NET DateTime value in a protocol buffers message.

My plan is to use the DateTime.ToBinary() and then pass the 64-bit returned value in the message. But I'm not sure what to choose as a protocol buffers data type to represent that.

I guess I'm confused about when the fixed64 (or sfixed64) data types should be used.

I'm assuming in this scenario I would use the signed types since the values returned by DateTime.ToBinary() can be negative as well as positive.

Answer 1

Well, you definitely want either int64 or sfixed64 to cope with the value being signed.

Having just done a quick test, DateTime.Now.ToBinary() is encoded in 10 bytes using int64 whereas sfixed64 will always use 8 bytes. Basically, the variable length encoding is great for small numbers, but becomes bigger than the fixed encoding for large numbers. (It's the same kind of tradeoff as using UTF-8 instead of UTF-16 - ASCII characters can be encoded in UTF-8 in a single byte, but later on code points end up being encoded as 2 and then 3 bytes, whereas UTF-16 always uses 2 bytes for characters in the BMP.)

My guess is that DateTime.ToBinary() values are likely to be quite large (without knowing the details of exactly what it does) so sfixed64 is more appropriate.

Does that make sense?

Answer 2

In protobuf-net, I use a graduated scale approach (and indeed, it handles all this for you if you simply use DateTime) - the equivalent .proto is something like this:

message DateTime {
  optional sint64 value = 1; // the offset (in units of the selected scale)
                             // from 1970/01/01
  optional TimeSpanScale scale = 2 [default = DAYS]; // the scale of the
                                                     // timespan
  enum TimeSpanScale {
    DAYS = 0;
    HOURS = 1;
    MINUTES = 2;
    SECONDS = 3;
    MILLISECONDS = 4;

    MINMAX = 15; // dubious
  }
}

i.e. if the DateTime can be expressed in whole days, I just send the number of days since 1970, etc - plus a small marker to the scale. This means that dates can be sent a bit more efficiently, but it doesn't really cost much more for other scales.

Personally, I wouldn't use ToBinary() - I would explicitly use an offset of a known scale from a known epoch (such as the unix epoch). This makes it more portable between platforms. But if you are sending (for example) just the millisecond offset, then a fixed scale would usually be more efficient than a variant-length scale. Whether you need signed or unsigned depends on whether you need dates before your epoch ;-p