Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is my Protobuf message (in Python) ignoring zero values?

I've been working on implementing protobufs for IPC for a project. For some reason, values that are set to 0 are not being set/serialized. For context, the .proto file contains the following message:

syntax = "proto3";

enum SetGet {
    SET = 0;
    GET = 1;
}

message State {
    SetGet setget = 1;
    double x = 2;
    double y = 3;
    double depth = 4;
    double yaw = 5;
    double pitch = 6;
    double roll = 7; 
}

I compile the file to a Python _pb2 file with protoc, and then I try running the following test script:

import filename_pb2 as pb

state = pb.State()
state.x = 0
state.y = 0
state.depth = 0
state.yaw = 0
state.pitch = 0
state.roll = 0
state.setget = pb.SET

print("State: {}".format(state))

state2 = pb.State()
state2.ParseFromString(state.SerializeToString())

print("State2: {}".format(state2))

When I run it, the following output is printed:

State: 
State2: 

It seems that nothing is being set, or that the zero values are somehow being ignored. However, when I change the values (x, y, depth, etc.) to something nonzero, say 0.1, I get the following, expected result:

State: x: 0.1
y: 0.1
depth: 0.1
yaw: 0.1
pitch: 0.1
roll: 0.1

State2: x: 0.1
y: 0.1
depth: 0.1
yaw: 0.1
pitch: 0.1
roll: 0.1

Even though the numbers are printed out, for some reason the enum still isn't. Why does this happen with protobufs? Is the double type 0 by default, so the protobuf serializer saves on space by ignoring them? Why, then, are they not being restored when State2 is parsed in? Is there some line in the documentation that I missed? Thanks in advance!

-- Tim

like image 630
Timothy Kanarsky Avatar asked Nov 19 '17 05:11

Timothy Kanarsky


People also ask

How do I set default value in Protobuf?

For bool s, the default value is false. For numeric types, the default value is zero. For enums , the default value is the first value listed in the enum's type definition. This means care must be taken when adding a value to the beginning of an enum value list.

Why are Protobuf fields numbered?

Field numbers are an important part of Protobuf. They're used to identify fields in the binary encoded data, which means they can't change from version to version of your service. The advantage is that backward compatibility and forward compatibility are possible.

How big can a Protobuf message be?

Protobuf has a hard limit of 2GB, because many implementations use 32-bit signed arithmetic. For security reasons, many implementations (especially the Google-provided ones) impose a size limit of 64MB by default, although you can increase this limit manually if you need to.

What is a message in Protobuf?

namespace google::protobuf. Defines Message, the abstract interface implemented by non-lite protocol message objects. Although it's possible to implement this interface manually, most users will use the protocol compiler to generate implementations.


2 Answers

Yes, 0 is the default. This case is mentioned explicitly in the documentation:

Note that for scalar message fields, once a message is parsed there's no way of telling whether a field was explicitly set to the default value (for example whether a boolean was set to false) or just not set at all: you should bear this in mind when defining your message types. For example, don't have a boolean that switches on some behaviour when set to false if you don't want that behaviour to also happen by default. Also note that if a scalar message field is set to its default, the value will not be serialized on the wire.

like image 72
amalloy Avatar answered Sep 25 '22 04:09

amalloy


Zero is the default for numerics in protobuf, and empty strings are defaults for strings. For efficiency, default values are not transmitted across the wire.

If you really want to be able to tell if it was set explicitly, don't use the default zero for a real operation:

enum SetGet {
    NONE = 0;
    SET = 1;
    GET = 2;
}

Just keep in mind this will result in extra traffic over the wire and, since you're only really worried out what's being printed, you can alternatively just understand that zeros are defaults, or write your own print routines that output everything.

like image 40
paxdiablo Avatar answered Sep 23 '22 04:09

paxdiablo