How to get Google Protobuf working in Matlab?

Question

So if one was to want to use Google protocol buffers in Matlab and using a Windows computer what would be the best way to do that since Matlab is not in the list of supported languages?

Answer 1

I haven't seen an answer on this and I thought the solution was a bit obscure so I am going to post a how to for matlab_out using the protoc.exe

A how to for google protocol buffer matlab out, this is using resources from the internet I will also include a zip file containing all this already done.

1. Unzip protobuf-‘version#’.zip (looks like: protobuf-#.#.#)

2. Open file protobuf-#.#.# -> src

3. Choose your favorite editor (Notepad++ is good) and change Makefile.am (do not include quotes)

a. Under “nobase_include_HEADERS =” and below “$(GZHEADERS)”

add the line “farsounder/protobuf/compiler/matlab/matlab_generator.h \” (Note the back slash)

b. Under “libprotoc_la_LIBADD = $(PTHREAD_LIBS) libprotobuf.la”

add lines
“farsounder/protobuf/compiler/matlab/matlab_generator.cc \”

“farsounder/protobuf/compiler/matlab/matlab_plugin.cc \”

c. Save the file and exit out

4. While still in the src directory go into ->google ->protobuf->compiler and change main.cc

a. Under “#include <google/protobuf/compiler/java/java_generator.h>”

add the line “#include <farsounder/protobuf/compiler/matlab/matlab_generator.h>”

b. In main function add the lines “// Proto2 Matlab farsounder::protobuf::compiler::matlab::MatlabGenerator matlab_generator; cli.RegisterGenerator("--matlab_out", &matlab_generator, "Generate Matlab M files.");”

c. Save the file and exit out

5. Unzip protobuf-matlab
6. Open file protobuf-matlab ->src and copy the farsounder directory to protobuf-#.#.# -> src directory
7. Go back to protobuf-#.#.# directory and into -> vsprojects and open protobuf.sln in Visual Studio, I believe any version should work
8. There should be a popup that wants to convert the solution file to a more updated version, go ahead and do that
9. If you do not see solutions explorer open it up using ctrl+alt+L
10. Open a new windows explorer and go into protobuf-#.#.# ->src->farsounder->protobuf->compiler->matlab, now in visual studio using the solutions explorer expand the libprotoc->Header Files, now in windows explorer copy the matlab_generator.h file and paste it into the Header Files directory
11. Still in Solutions explorer go into the Source Files directory and copy in the matlab_generator.cc
12. Contract libprotoc and right click on protoc and click on properties
13. Under Configuration Properties->Linker->General, Edit Enable Incremental Linking to be No
14. Under Configuration Properties->Linker->Input, Edit Additional Dependencies a. Add the lines “Release\libprotobuf.lib” and underneath “Release\libprotoc.lib” b. Click OK on the Additional Dependencies window and OK on the protoc Property Page
15. Change the build type to Release
16. Right-click on libprotobuf and select build, once completed right-click on libprotoc and select build
17. Once completed right-click on protoc and select build, it should provide you with an protoc.exe under protobuf-#.#.# ->vsprojects->Release, this will now allow you to create matlab .m files by using the matlab_out command

Help to find protobuf-#.#.# and the protobuf-matlab:

Follow these links:

https://code.google.com/p/protobuf-matlab/source/browse/
download the zip file

https://github.com/google/protobuf/releases

download the source code

Extra help, for using protoc.exe

1. Copy and paste the created protoc.exe to where your .proto file is
2. Run cmd and change the directory to where your protoc.exe and .proto file is
3. Run this command (lets pretend the name of my .proto file is “afunprotofile”) “protoc –matlab_out=./ -I./ afunprotofile.proto”

Answer 2

The easiest way is to use Java

The FarSounder code was nice, but it is quite old and unmaintained. The easiest way to produce Matlab compatible code is to just use the Java version of Protobuf. This solution should work on any platform that supports Matlab and Java.

The Steps

1. Create a .proto file
2. Process the file with the protoc compiler and output Java source
3. Using IntelliJ or other tool produce a JAR file that includes the dependencies
4. Add the JAR file to the Matlab classpath. edit('classpath.txt')
5. Restart Matlab

The protobuf runtime dependencies

I include these in a single JAR file output of the Protobuf and the two runtime libraries.

• protobuf-java-3.3.0.jar
• protobuf-java-util-3.3.0.jar

A sample

I wrote a simple Java wrapper class to hide the MyProtobuf.Builder return type from Matlab that I added to the JAR file

public class MyProtobufWrapperWrapper {
    public static MyProtobuf.Builder newBuilder()
    {
        return MyProtobuf.newBuilder();
    }
}

In Matlab

p = com.cameronpalmer.MyProtobufWrapper.newBuilder();
p.setIdentifier(java.util.UUID.randomUUID().toString());
p.setTimestampMilliseconds(timestamp);
p.setAngleRadians(0);
p.addAllChannelSamples(channel_vector);

planeWaveBuilt = p.build();
byteArray = planeWaveBuilt.toByteArray();

Answer 3

As Cameron Lowell Palmer's answer suggests, the way to go is via Java.

Is I lost a couple of hours today on this problem, I would like to elaborate some more. I started with Cameron's answer but I had to do a couple of more steps. Essentially, I had to do all of the following:

1. Compile proto message for java, e.g. protoc --java_out=./ your_file.proto
2. Remember to install protobuf java runtime, e.g. in ubuntu: sudo apt install libprotobuf-java
3. Locate the protobuf java runtime on your system, e.g.: /usr/share/java/protobuf-3.6.1.jar; its path will be used later on. The name should always follow the pattern protobuf-version.jar or protobuf-java-version.jar, therefore locate protobuf- | grep jar$ should reduce the search space for you.
4. Add a short java file to the same directory, where your protobuf java files went (./x/y/z/MyProto.java). Use your package path instead of x/y/z. If you did not declare java package explicitly in the proto file, then protoc just used your filename as package name. Either way, you can probably check where the protoc generated files went by yourself. Contents of MyProto.java are listed below. Just replace YourProtoFileName and YourMessageName with your stuff. Note that this step is not optional, as this will not generate a simple helper class. For the life of me, Matlab would not let me use inner classes directly (in java, YourMessageName is the inner class of YourProtoFileName). But with the above helper, it was quite happy to generate them for me. Note that if you have more than one message defined in your proto file, you might need to expose more than one builder in this way. And if you only need to read protobuffers, then you might need to export just YourMessageName and not Builder.

package x.y.z;

import x.y.z.YourProtoFileName; // if you do not know it, do `ls x.y.z/*.java`

public class MyProto {
   public static YourProtoFileName.YourMessageName.Builder newBuilder() {
      return YourProtoFileName.YourMessageName.newBuilder();
   }
}

5. Check Matlab's version of java; in Matlab write version -java. In my case it was 1.8, while the default java installed on my system (java -version) is 11. I had to manually select java 1.8 for the next step, otherwise the whole thing did not work. Even worse, Matlab only produced a very nondescript error "No class x.y.z.YourProtoFileName." Thanks Matlab! You might need to install proper version first (sudo apt install openjdk-8-jdk) and then use update-java-alternatives or just locate javac for the appropriate java version on your system.
6. Compile both your java file and the one generated by protoc. Remember to point classpath at the jar file or protobuffer runtime that you found above. And remember to use your javac path. In a single command: /usr/lib/jvm/java-8-openjdk-amd64/bin/javac x.y.z/*.java -cp /usr/share/java/protobuf-3.6.1.jar. This will generate class files in ./x/y/z/.
7. Optional: pack the class files in a jar for easier distribution: jar cvf ./YourProtoFilename.jar x/y/z/*.class. Note that this command line works for me, since I've put all classes, including MyProto, in the same package. You might need to adapt it to your needs.
8. Start Matlab.

% make Matlab aware of your new classes
javaaddpath('./')
% tell Matlab where protobuf dependancy lives (use the path from step 3)
javaaddpath('/usr/share/java/protobuf-3.6.1.jar')

% test if the classes were found
methods('x.y.z.YourProtoFilename.YourMessageName')
% if methods are listed then you are good to go

% use the helper form step 4
b = x.y.z.MyProto.newBuilder();
% now you have a builder you can use to build your protobuf message

9. See Google's protobuf java primer to go on from here, as using java in Matlab is fairly straight forward: just write java statements. As long as they are simple statements :)

The same procedure works on octave too. With a bit different syntax for java inside octave. And octave was less picky about the java version in my case. YMMV