Protobuf defaults vs. "missing required fields" when serializing

Question

The SerializeToString() method on a ProtoBuf message containing required fields, that have defaults, always throws the EncodeError stating that the message is missing those required fields. However if I check the value of the fields, all defaults are set. For example:

// mymessage.proto
message MyMessage {
    required int32 val = 1 [default=18];
}

Then in python:

from mymessage_pb2.py import MyMessage
m = MyMessage()
print m.val # Shows m.val == 18
print m.SerializeToString() # EncodeError

On the other hand if I do this:

m.val = m.val
print m.SerializeToString() # No Error

So clearly, it's just needing each field to get touched despite having a default value on initialization. For me, one of the major points of having defaults was to only require someone to update the non-default fields (or those fields they need to change), so this set-it-to-itself method is a pretty sorry solution.

Marking the fields as optional is not a solution since these fields are legitimately required per our spec.

UPDATE: My attempt at a work-around included MergeFrom and CopyFrom, but neither worked. So I wrote this:

def ActuallyInit(obj):
    err = []
    obj.IsInitialized(err)
    for field in err:
        attr = obj.__getattribute__(field)
        try:
            obj.__setattr__(field, attr)
        except:
            ActuallyInit(attr)

Then one creates their protobuf object and passes it into ActuallyInit which recursively sets each field to itself. This seems like an ugly hack, so I'm leaving the question below open.

QUESTION: Is there a way to create the ProtoBuf message instance and "convince" it that every already-initialized-to-default field is in-fact not an error?

Answer 1

This is working as intended. required means "the writer must explicitly fill in this field, not use the default". If you want a field where the writer is allowed to leave the field with its default value, then you want optional. This is in fact the only difference between required and optional, so there's simply no reason to use required with default values. Arguably, the Protobuf compiler should raise an error if a required field is defined with a default value, but I didn't think to implement that restriction at the time.

(FWIW, required has long been considered a misfeature and has been removed in Protobuf v3.)