Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Importing protocol buffer definitions between Maven projects

I currently manage a few separate Maven projects in which I use Protobufs as a serialization format and over the wire. I am using David Trott's maven-protoc plugin to generate the code at compile time.

All is good and well until I want those project to communicate between one another — or rather, use each other's protobufs. The protobuf language has an "import" directive which does what I want but I'm faced with the challenge of having project A exporting a ".proto" file (or possibly some intermediate format?) for project B to depend upon.

Maven provides a way for a project to bundle resources but AFAIK, these are meant to be used at runtime by the code and not by a goal during the compile / source generation phase — at least I haven't been able to find documentation that describes what I want to achieve.

like image 236
Julien Silland Avatar asked Oct 31 '12 18:10

Julien Silland


People also ask

What is Protobuf Maven plugin?

Maven Protocol Buffers Plugin uses Protocol Buffer Compiler (protoc) tool to generate Java source files from . proto (protocol buffer definition) files for the specified project. For more information about the Protocol Buffer Compiler, please refer to Reference Guide.

What is a proto file?

A . proto file is similar to a JSON file in that it represents structured data, but you can run a compiler on the . proto file to generate code that can read and write the data in the programming language of your choice. For more information about protocol buffers, see Protocol Buffer Developer Guide on Google's site.

How does the Maven protocol buffer plugin work?

Maven Protocol Buffers Plugin generates Java, C++ or Python sources from .proto files using the protoc tool. The following examples describe the basic usage of the Plugin.

How do I add a protocol buffer in protoccompiler?

Open the pom file at protoccompiler/pom.xml, look for the “dependencies” element, and add the following dependency to pull in protocol buffer support files. Now look for the “plugins” element and add the plugin definition below.

Is there a native Java compiler for Protocol Buffers?

Java build systems have always been a second class citizen for Protocol Buffers. As is the case for many other Java serialization frameworks (e.g., Cap’n Proto, FlatBuffers), Protocol Buffers does not provide a native Java compiler which you can inject into Maven dependencies and invoke a plugin to compile .protofiles into .javasources.

What are Protobuf buffers?

Protocol Buffers (protobuf) is a method of serializing structured data which is particulary useful to communication between services or storing data. It was designed by Google early 2001 (but only publicly released in 2008) to be smaller and faster than XML.


Video Answer


2 Answers

I've found another way to achieve, and it doesn't involve any Maven magic. Diving into the code for the maven-protoc plugin, I found that this is a supported use case -- the plugin will look for and collect and .proto files in dependent jars and unpack them into a temporary directory. That directory is then set as an import path to the protoc invocation.

All that needs to happen is for the .proto file to be included in the dependency's package, which I did by making it a resource:

projects/a/src/main/resources/a.proto

Now in projects/b/pom.xml, add 'a' as a regular Maven dependency and just import a.proto from b.proto as if it existed locally:

b.proto: import "a.proto";

This isn't ideal, since files names may clash between various projects, but this should occur rarely enough.

like image 181
Julien Silland Avatar answered Oct 16 '22 19:10

Julien Silland


You can package your .proto files in a separate .jar/.zip in the project where they are generated, and publish them in your repository using a dedicated classifier. Using the assembly plugin might help here to publish something close to "source jars" that are built during releases.

Then, on projects using them, add previously created artifact as dependency. Use the dependency plugin with the "unpack-dependencies" goal, and bind it to a phase before "compile".

like image 20
blindpotatoes Avatar answered Oct 16 '22 17:10

blindpotatoes