does protobuf need a network packet header?

Question

I am using 'protobuf' for C/S network program using TCP.

here is my steps for client:

1, pack data into a 'protobuf'

2, get size in bytes of the pack and construct a length-prefix frame

3, write the frame+pack to socket

and then the server:

1, read Length-prefix frame from socket and get the length N

2, read N bytes from socket and fill the data into protobuf instance

3, get "value"s from protobuf by "key"s

it seems a little complicated I think, is there some kind of auto-generated length-prefix frame for protobuf, then I do not need construct one by myself. or anything else I can do to make code cleaner?

Answer 1

With protobuf, and assuming you are sending multiple messages down the same pipe, then: yes - you will need to have a length as a prefix, otherwise it will want to read to the end of the stream.

protobuf does include some basic RPC stubs, but the RPC implementation they use is not part of the OSS project, so that is not available. There are some standalone protobuf RPC stacks listed in the implementations though.

Personally, I tend to pretend that the sequence of data is part of a repeated sequence - i.e. prefix with "field 1, string" (aka 0a), and the length as "varint" encoded. This means that the entire network stream is a valid protobuf stream. That might just be my OCD kicking in, though.

Some implementations may have features included to help with this. protobuf-net (one of the .NET versions) has SerializeWithLengthPrefix/DeserializeWithLengthPrefix methods, for example, which lets the library do this for you (while offering a range of formats for your choosing).