Optional fields and constraints in Protocol Buffers (protobuf) v3.0.0-alpha-2

Question

I am currently playing around with v3.0.0-alpha-2 of Google Protocol Buffers.

As far as I understand v3 removes the required keyword, the extensions keyword and default values for fields to simplify the proto language.

What I do not understand is the meaning of the optional keyword in proto3.

Example:

syntax = "proto3";

package fw.example;

message ExampleMessage {
  optional string optional_string = 1;
  string normal_string = 2;
}

Question: What is the difference between optional_string and normal_string besides the name and the tag?

I've already read the following resources (they seem to be the only publicly available yet for v3 of protobuf):

• Protocol Buffers v3.0.0-alpha-2 Release Notes
• proto3 Language Guide

But they do not even mention the optional keyword.

• Is optional obsolete in proto3, since a field is always optional?
• How can one enforce required fields with proto3 if required is gone?

It seems that in proto3 one cannot longer distinguish between unset fields and fields set by a client to the (implicit) default value.

Is the best practice to wrap each proto3 message in a language-specific class? I am using C++ and I need to ensure that specific fields are set. It seems that validation has to be done manually in language-specific source code now, in contrast to proto2.

Can someone enlighten me, what's the best approach to apply constraints on a proto3 message but allow scheme evolution? At the moment I think that a new API has to be written around the proto3 messages, so that a client does not deal directly with the proto3 generated code, but with the custom API code. Is that correct?

Maybe someone can show me a concrete example to discuss about.

I am pretty confused, since the following is stated in the release notes of v3:

We recommend that new Protocol Buffers users use proto3. However, we do not generally recommend that existing users migrate from proto2 from proto3 due to API incompatibility, and we will continue to support proto2 for a long time.

If proto3 is the way-to-go, why are things complicated? It seems to me that I do need to write a lot more code than with proto2 now.

Answer 1

My reasoning is as follows:

Since required is obsolete, then everything is optional. Thus there is no reason for explicit keyword.

The following paragraph from language guide indicates how values are initialized if not set:

Note that for scalar message fields, once a message is parsed there's no way of telling whether a field was explicitly set to the default value (for example whether a boolean was set to false) or just not set at all: you should bear this in mind when defining your message types.

So optional really means "if you didn't set it explicitly, we'll set it to default value for you!"

Note that this allows them to do some nice optimization (not include the value on the wire if it's set to default):

For example, don't have a boolean that switches on some behaviour when set to false if you don't want that behaviour to also happen by default. Also note that if a scalar message field is set to its default, the value will not be serialized on the wire.