Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Do not set default values in optional protobuf fields to minimize data sent over the wire

Should we not set default values in optional protobuf fields to minimize data sent over the wire?

I want to minimize my message byte size sent over the wire. To accomplish this, one optimization which I could think of was:

if( message->my_optional_field() != value )
  message->set_my_optional_field(value);

This prevents has_my_optional_field() to be called when the intended value is equal to the default value and hence, prevent the field to appear in the serialization array. Is this a good practice? Does protobuf provide something like this out of the box?

The question is similar to how do has_field() methods relate to default values in protobuf? In fact, it has been answered in one of the comments to the accepted answer. However, subsequent comments have disputed the claim.

like image 213
Mukul Gupta Avatar asked May 04 '16 17:05

Mukul Gupta


People also ask

Are proto fields optional by default?

As of Google Protobuf version 3.15, optional fields are reintroduced in proto3. Embedded Proto supports optional fields as of version 2.3.

Are all fields optional in proto3?

proto3 basically makes all fields optional.

Why did proto3 remove optional?

We have seen production issues caused by this multiple times and it's pretty much banned everywhere inside Google for anyone to add/remove required fields. For this reason we completely removed required fields in proto3. After the removal of "required", "optional" is just redundant so we removed "optional" as well.


1 Answers

The behavior differs between proto2 and proto3.

Under proto2, the concept of "is present" was separate from the concept of default values. Setting a field to its default value in proto2 is not the same as clearing it; the value would be sent over the wire even though it was the default. In proto2, in order to "unset" the field you had to call clear_my_optional_field(); then it would not be sent on the wire. Proto3 also had the separate has_my_optional_field() method to check whether or not the field was set.

Under proto3, the concept of "is present" has been removed. Instead, a field will be sent if and only if it is not set to the default value. The clear_ method is the same as setting to default. The has_ method no longer exists. Additionally, proto3 did away with the notion of configurable default values -- the default for all fields is 0 or empty.

Exception: For message-typed fields, the behavior hasn't changed in proto3. There is still a concept of presence for these fields.

like image 167
Kenton Varda Avatar answered Oct 16 '22 13:10

Kenton Varda