Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does google protocol buffers support size calculation before serialization?

My (DSP) application produces data at a constant rate. The rate depends on the configuration that is selected by the user. I would like to know how many bytes are generated per second. The data structure contains a repeated (packed) floating point field. The length of the field is constant, but can be changed by the user.

Is there a protocol buffers function that will calculate the message size before serialization?

like image 238
Dan Avatar asked Nov 15 '10 14:11

Dan


People also ask

How do Google Protocol Buffers work?

Protocol buffers provide a language-neutral, platform-neutral, extensible mechanism for serializing structured data in a forward-compatible and backward-compatible way. It's like JSON, except it's smaller and faster, and it generates native language bindings.

Is Protobuf more efficient than JSON?

JSON is usually easier to debug (the serialized format is human-readable) and easier to work with (no need to define message types, compile them, install additional libraries, etc.). Protobuf, on the other hand, usually compresses data better and has built-in protocol documentation via the schema.

What are Protocol Buffers used for?

Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data. It is useful in developing programs to communicate with each other over a network or for storing data.

How is Protobuf serialized?

The Protobuf serialization mechanism is given through the protoc application, this compiler will parse the . proto file and will generate as output, source files according to the configured language by its arguments, in this case, C++. You can also obtain more information about, reading the section compiler invocation.


2 Answers

If you have build the message objects, you can call ByteSize() on the message which returns the number of bytes the serializes message would take up. There is a link to the C++ docs of ByteSize.

like image 125
dmeister Avatar answered Oct 05 '22 06:10

dmeister


It's impossible to know ahead of time, because protobuf packs the structures it is given into the fewest bytes possible - it won't use four bytes for int x = 1; for example - so the library would have to walk the entire graph to know the output size.

I believe you could find this out by doing a serialize operation to a protobuf-compliant stream of your own design that just counts the bytes it is given. That could be costly, but no more costly than it would be for the library to do that work.

like image 33
Steve Townsend Avatar answered Oct 05 '22 07:10

Steve Townsend