Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can a proto3 optional field be changed to repeated without breaking wire compatibility?

Let's say that I have a proto3 message defined as follows, for use as a gRPC request (i.e. using protobuf's binary encoding):

message MyRequest {
  string name = 1;
}

Can I change my server (i.e. the reader of the message) to use the following definition without breaking wire compatibility for existing clients (i.e. writers)?

message MyRequest {
  repeated string names = 1;
}

In the proto2 language guide, I see the following:

optional is compatible with repeated. Given serialized data of a repeated field as input, clients that expect this field to be optional will take the last input value if it's a primitive type field or merge all input elements if it's a message type field.

However, the proto3 documentation does not contain an equivalent statement. I think that this may be related to the use of the packed encoding for repeated fields in proto3.

like image 845
paulkernfeld Avatar asked Aug 23 '17 21:08

paulkernfeld


People also ask

Is repeated optional Protobuf?

You don't need the optional modifier, and it looks like it is confusing the parser. A repeated field is inherently optional : you just don't add any values. As for com.

Is proto3 backwards compatible?

One of selling points of Protobuf was backward compatibility, i.e. developers can evolve format, and older clients can still use it. Now with new Protobuf version called proto3, the IDL language itself is not compatible as such things as options , required where dropped, new syntax for enuns, no extention.

Are repeated fields ordered in Protobuf?

Yes, repeated fields retain the order of items.

Why did proto3 remove optional?

We have seen production issues caused by this multiple times and it's pretty much banned everywhere inside Google for anyone to add/remove required fields. For this reason we completely removed required fields in proto3. After the removal of "required", "optional" is just redundant so we removed "optional" as well.


1 Answers

Yes, this is possible as the binary encoding for an optional string and for a repeated string with a single element are the same. However, this change may be confusing to readers of the code because it is not immediately obvious that a message can be reinterpreted in this way.

like image 174
paulkernfeld Avatar answered Sep 28 '22 08:09

paulkernfeld