Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Data format compatibility between protobuf versions

I was wondering if protocol buffer's serialized data format remains constant across protobuf compiler and client library versions.

In other words, do I need to use the same compiler version to generate my Python, Java, and C++ classes? And do these clients all need to use the same version of protobuf libraries?

This post sort of addresses my question, but its accepted answer is specific to the OP's protobuf version.

like image 325
pepsi Avatar asked Jul 07 '11 01:07

pepsi


People also ask

Is protobuf backwards compatible?

Protocol buffers provide a language-neutral, platform-neutral, extensible mechanism for serializing structured data in a forward-compatible and backward-compatible way.

Why is protobuf backwards compatible?

It can accept input crafted by later versions of protobuf. The sender is backward compatible because it's creating output that can be consumed by earlier versions. So long as you're careful about when and how you change and remove fields, your protobuf will be forward and backward compatible.

Is proto3 backwards compatible with proto2?

Using proto2 Message TypesIt's possible to import proto2 message types and use them in your proto3 messages, and vice versa. However, proto2 enums cannot be used directly in proto3 syntax (it's okay if an imported proto2 message uses them).

What format is protobuf?

Protocol buffers, or Protobuf, is a binary format created by Google to serialize data between different services. Google made this protocol open source and now it provides support, out of the box, to the most common languages, like JavaScript, Java, C#, Ruby and others.


1 Answers

Yes, that is pretty much the idea. It shouldn't matter which library you use, as long as it follows the spec. Note that the same data can be represented in slightly different ways, for example field order should not matter to the client, and while it is common for clients to write fields in ascending order, it is explicitly required of clients to process fields in any order. All I'm saying here is that it might not be exactly the same bytes in the same order, but it should work fine from any client.

Note that some implementations might offer additional features (*cough* like mine offers inheritance support), intended for use primarily only within that single client. In that case, I would a: expect those features to be obvious when used, and b: it should always still produce a valid protobuf stream (you might just choose to ignore those fields, or support them as bytes for the purpose of round-trip).

like image 192
Marc Gravell Avatar answered Jan 04 '23 08:01

Marc Gravell