Upgrading protobuf from version 2 to 3 - incompatible with protobuf default values

Question

I'm trying to upgrade to using protobuf version 3, and stay backwards compatible with version 2. Seems to work except for one thing - in proto-2 you could set your own default values, but in proto 3, you can't. If you chose a default value in proto-2 that is not the standard default value in proto-3, then you have a problem. For example, in proto-2:

message Record {   required uint32 fileno = 1;                  required uint64 pos = 2;                     optional uint64 bmsPos = 3 [default = 0];    optional uint32 scanMode = 4 [default = 9999];   } 

now in proto-3 must be:

message Record {   uint32 fileno = 1;                  uint64 pos = 2;                     uint64 bmsPos = 3;    uint32 scanMode = 4;   } 

In both proto-2 and proto-3, missing values aren't sent in the message. But the proto-3 API doesn't tell you if the default value is in the message or not, it just tells you the value.

So the proto-3 receiver gets a message and tells me that scanMode = 0. If that message came from a proto-2 sender, then either 1) the proto-2 sender placed a 0 in the message, or 2) the proto-2 sender set the value to 9999 (the default value), and so the value is not sent, and the proto-3 receiver interprets it as a 0. Without knowing if the value is present in the message or not, my code can't disambiguate, even if it knows whether the message came from a proto-2 or proto-3 sender.

Note that there's no problem with the bmsPos field in the example, since the proto-2 message uses the same default value as proto-3 (0). But if you happened to have chosen a default value not the same as proto-3, then I don't see how to upgrade to proto-3 and be backwards compatible.

Answer 1

Turns out there is a way to find out if a default value is actually missing or not (thanks to some friends at google for this answer):

message Record {   uint32 fileno = 1;                  uint64 pos = 2;                     uint64 bmsPos = 3;    oneof scanMode_present {     uint32 scanMode = 4;   }   uint32 version = 5; // set to >= 3 for protobuf 3  } 

The generate code has additional methods to detect if oneof fields are set, using the getXXXcase() method:

int scanMode = proto.getScanMode(); boolean isMissing = proto.getScanModePresentCase() == Record.ScanModePresentCase.SCANMODEPRESENT_NOT_SET; if (isMissing) {   boolean isProto3 = proto.getVersion() >= 3;   scanMode = (isProto3) ? 0 : 9999; } 

• Note that the name of the oneof is arbitrary, i have adopted the convention fieldname_present.
• The oneof doesnt add anything to the wire format, so it stays compatible with the proto-2 messages.
• You can add the version info anywhere that makes sense, i put it in the Record message for this example.

With this 'trick', i have upgraded to proto-3 with backwards compatibility with non-standard proto-2 default values.