Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can you statically link the same protobuf message into multiple DLLs - and then have those DLLs work together?

We are using google protobufs to pass data around on the wire. The server side of things is plugin-like so several of the modules handling the protobuf messages are DLLs. Some DLLs depend on others and use the others' messages to define their own messages.

So, A.DLL has a.proto which generates a.pb.h/cc with a message class MsgA. Using the undocumented dllexport_decl option on the protoc compiler, the message class is declared as a DLL export.

Now, B.DLL depends on A.DLL, and b.proto also looks like this:

import "a.proto";
message b 
{
    required int32 some_number = 1;
    required PackageA.MsgA some_a = 2;
}

Finally, the executable that pulls the parts together also depends on the message MsgA. The protobuf library was also built as a DLL and is linked into everything. It all builds and runs.

But, there are Forces of Light which have demanded that we cut down on the DLL distribution! So, I built the module A (which is just a collection of messages and small utils that a lot of the other plugin DLLs use) as a static library instead of a DLL. B.DLL and the executable both link to A and everything is good - so far.

Since A is statically linked, MsgA gets fully defined in all the DLLs and the EXE. That's OK because all the static data in the generated C++ code is const. So what if there are multiple copies in the final process - all of the copies are identical.

But, when I run the newly built process, libproto throws an actually useful exception - MsgA's file ID already exists in the descriptor map (or something like that). In other words, the fact that there are multiple definitions for MsgA is a major problem.

So, finally, here is the question(s):

  • If we statically linked the libproto instead of using it as a DLL, would the error go away?
  • Is it really safe to have the multiple definitions of MsgA scattered throughout the DLLs?

The first point I can probably answer myself in a few days once I get around to rebuilding the protobuf libraries, but the second one is a little further beyond my current knowledge. I'm also hoping to get a quick up or down answer that may save me the trouble of recompiling the proto libs.

like image 522
Daver Avatar asked Feb 24 '10 22:02

Daver


1 Answers

I've used protobuffers for RPC across a network (Google does this too - see the documentation page). As long as you have a similar definition for everyone using the proto buffer, one definition will happily deserialize data serialized by other definitions. In fact, as long as you don't re-assign the tag numbers, older versions of the protocol buffer definition can interact with newer versions just fine (as long as "required" fields in the deserializing definition exist in the stream, it will succeed).

Hope that helps.

like image 73
Smokey.Canoe Avatar answered Oct 12 '22 23:10

Smokey.Canoe