Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Convert json to dynamically generated protobuf in Java

Given the following json response:

{
    "id" : "123456",
    "name" : "John Doe",
    "email" : "[email protected]"
}

And the following user.proto file:

message User {
    string id = 1;
    string name = 2;
    string email = 3;
}

I would like to have the possibility to dynamically create the protobuf message class (compile a .proto at runtime), so that if the json response gets enhanced with a field "phone" : "+1234567890" I could just upload a new version of the protobuf file to contain string phone = 4 and get that field exposed in the protobuf response, without a service restart.

If I were to pull these classes from a hat, I would like to be able to write something along the following code.

import com.googlecode.protobuf.format.JsonFormat;
import com.googlecode.protobuf.Message;
import org.apache.commons.io.FileUtils;

...

public Message convertToProto(InputStream jsonInputStream){
    // get the latest user.proto file
    String userProtoFile = FileUtils.readFileToString("user.proto");

    Message userProtoMessage = com.acme.ProtobufUtils.compile(userProtoFile);
    Message.Builder builder = userProtoMessage.newBuilderForType(); 
    new JsonFormat().merge(jsonInputStream, Charset.forName("UTF-8"), builder);
    return builder.build();
}

Is there an existing com.acme.ProtobufUtils.compile(...) method? Or how to implement one? Running a protoc + load class seems overkill, but I'm willing to use it if no other option...

like image 780
Alin Stoian Avatar asked Aug 23 '18 08:08

Alin Stoian


People also ask

How is protobuf faster than JSON?

JSON is usually easier to debug (the serialized format is human-readable) and easier to work with (no need to define message types, compile them, install additional libraries, etc.). Protobuf, on the other hand, usually compresses data better and has built-in protocol documentation via the schema.

Does protobuf support JSON?

Utility classes to convert protobuf messages to/from JSON format. The JSON format follows Proto3 JSON specification and only proto3 features are supported.

What is .proto file in Java?

proto file. The definitions in a . proto file are simple: you add a message for each data structure you want to serialize, then specify a name and a type for each field in the message. Here is the . proto file that defines your messages, addressbook.

How do I create a .proto file?

You can use any text editor to create a . proto file. If you'd like to have syntax highlighting there are also editors that would give you that. I use IntelliJ but a quick Google search found an Eclipse plugin which appears to be free: https://code.google.com/p/protobuf-dt/.


1 Answers

You cannot compile the .proto file (at least not in Java), however you can pre-compile the .proto into a descriptor .desc

protoc --descriptor_set_out=user.desc user.proto

and then use the DynamicMessage's parser:

DynamicMessage.parseFrom(Descriptors.Descriptor type, byte[] data)

Source: google groups thread

like image 154
MaanooAk Avatar answered Oct 20 '22 15:10

MaanooAk