Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Length prefix for protobuf messages in C++

I'm using protobuf to serialize message I send over a socket connection in C++. For the communication I'd like to add a header to the messags indicating the length of the message. What do you think about this implementation? I did some research and that's what I put together.

Is there any nicer way to do this? Can this implementation cause any trouble? I know that there is API support for Java, but unfortunately not for C++.

bool send_message(int socket, my_protobuf::Message message)
{
  google::protobuf::uint32 message_length = message.ByteSize();
  int prefix_length = sizeof(message_length);
  int buffer_length = prefix_length + message_length;
  google::protobuf::uint8 buffer[buffer_length];

  google::protobuf::io::ArrayOutputStream array_output(buffer, buffer_length);
  google::protobuf::io::CodedOutputStream coded_output(&array_output);

  coded_output.WriteLittleEndian32(message_length);
  message.SerializeToCodedStream(&coded_output);

  int sent_bytes = write(socket, buffer, buffer_length);
  if (sent_bytes != buffer_length) {
    return false;
  }

  return true;
}

bool recv_message(int socket, my_protobuf::Message *message)
{
  google::protobuf::uint32 message_length;
  int prefix_length = sizeof(message_length);
  google::protobuf::uint8 prefix[prefix_length];

  if (prefix_length != read(socket, prefix, prefix_length)) {
    return false;
  }
  google::protobuf::io::CodedInputStream::ReadLittleEndian32FromArray(prefix,
      &message_length);

  google::protobuf::uint8 buffer[message_length];
  if (message_length != read(socket, buffer, message_length)) {
    return false;
  }
  google::protobuf::io::ArrayInputStream array_input(buffer, message_length);
  google::protobuf::io::CodedInputStream coded_input(&array_input);

  if (!message->ParseFromCodedStream(&coded_input)) {
    return false;
  }

  return true;
}
like image 917
multiholle Avatar asked Jul 24 '12 23:07

multiholle


People also ask

How big can a protobuf message be?

Protobuf has a hard limit of 2GB, because many implementations use 32-bit signed arithmetic. For security reasons, many implementations (especially the Google-provided ones) impose a size limit of 64MB by default, although you can increase this limit manually if you need to.

What are the numbers in protobuf?

Field numbers are an important part of Protobuf. They're used to identify fields in the binary encoded data, which means they can't change from version to version of your service. The advantage is that backward compatibility and forward compatibility are possible.


1 Answers

It's more common to use varint (e.g. WriteVarint32) rather than fixed32 (where you have WriteLittleEndian32), but the practice of delimiting protobuf streams by prefixing with length is sound.

like image 82
ephemient Avatar answered Sep 19 '22 10:09

ephemient