Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Self referencing fields in protobuf messages

Is it ok to have message like this?

message A {
  required int64 some_number = 1;
  // .... some more fields
  optional A sub_a = 123;
}

The reason is my current protocol stores set of A directly, and wrapping A's in another message will lead to massive conversions of stored data.

2.2.0 protoc compiles it ok. Can this make any problems with serialization/deserialization, and is it supported by protobuf-net.

like image 421
float_dublin Avatar asked May 10 '11 04:05

float_dublin


People also ask

Why are Protobuf fields numbered?

Field numbers are an important part of Protobuf. They're used to identify fields in the binary encoded data, which means they can't change from version to version of your service. The advantage is that backward compatibility and forward compatibility are possible.

Can you remove field from Protobuf?

Removing fields is fine, although you might want to mark it reserved so that nobody reuses it in an incompatible way. New code with old data (with the field) will silently ignore it; old code with new data will just load without the field populated, since everything in proto3 is implicitly optional .

How do I set default value in Protobuf?

For bool s, the default value is false. For numeric types, the default value is zero. For enums , the default value is the first value listed in the enum's type definition. This means care must be taken when adding a value to the beginning of an enum value list.

Can proto3 import proto2?

It's possible to import proto3 message types and use them in your proto2 messages, and vice versa. However, proto2 enums cannot be used in proto3 syntax.


2 Answers

That is a perfectly fine definition, and should work in any implementation (including protobuf-net); are you seeing any problem? HOWEVER! you might want to consider the computational impact of serialization - in particular, to serialize a sub-message, the size of the sub-messages needs to be known first. A deeply recursive method (as necessitated by this linked-list) may cause some issues.

Is there any reason this can't be just a repeated message instead? that would by far be my preference.

like image 80
Marc Gravell Avatar answered Sep 21 '22 05:09

Marc Gravell


I don't know about protobuf-net, but it should be absolutely fine. I suspect that if it doesn't work in protobuf-net, Marc would count that as a bug, and fix it... that's certainly the attitude I'd take in my C# port :)

(Realistically, I can't easily see how it would be a problem... it's not like messages are going to be represented by structs, where the recursion would be an issue.)

It should be pretty easy to test - I suggest you give it a try with a small message, and see if you run into any problems. All you've really got to do is create a message and test whether you can serialize and deserialize it correctly, possibly between different platforms.

EDIT: Obviously you need to make sure there are no actual cycles in terms of the messages themselves...

like image 45
Jon Skeet Avatar answered Sep 18 '22 05:09

Jon Skeet