Is there a way to write proto data without code generation?

Question

I would like to know if it's possible to use the reflection API provided by google protobuf to serialize messages without code generation?

Protocol buffer allows us to use reflection on Message or Message.Builder objects after parsing process. But in my case, I would like to know if I can fill these object with fields/values, then write them into a file.

Answer 1

CodedOutputStream

One way of doing this is by understanding how a message is encoded and use the CodedOutputStream to write message fields with the appropriate write*() methods.

E.g. to write the following message:

message MyMessage {
    int foo = 1;
    string bar = 2;
}

You would use this piece of code:

ByteArrayOutputStream baos = new ByteArrayOutputStream();
CodedOutputStream out = CodedOutputStream.newInstance(baos);
out.writeInt32(1, 1);
out.writeString(2, "s");
out.flush();
byte[] rawProtocolBuffer = baos.toByteArray();

DynamicMessage

Another way is to create the descriptors by hand and then use DynamicMessage to set the respective fields, but this is more boilerplate than using CodedOutputStream directly.

String messageName = "MyMessage";
FileDescriptorProto fileDescriptorProto = FileDescriptorProto
        .newBuilder()
        .addMessageType(DescriptorProto.newBuilder()
                .setName(messageName)
                .addField(FieldDescriptorProto.newBuilder()
                        .setName("foo")
                        .setNumber(1)
                        .setType(FieldDescriptorProto.Type.TYPE_INT32)
                        .build())
                .addField(FieldDescriptorProto.newBuilder()
                        .setName("bar")
                        .setNumber(2)
                        .setType(FieldDescriptorProto.Type.TYPE_STRING)
                        .build())
                .build())
        .build();

Descriptor messageDescriptor = FileDescriptor
        .buildFrom(fileDescriptorProto, new FileDescriptor[0])
        .findMessageTypeByName(messageName);

DynamicMessage message = DynamicMessage
        .newBuilder(messageDescriptor)
        .setField(messageDescriptor.findFieldByNumber(1), 1)
        .setField(messageDescriptor.findFieldByName("bar"), "s")
        .build();

byte[] rawProtocolBuffer = message.toByteArray();

Answer 2

First you need to compile the proto file into a desc file.

protoc --descriptor_set_out=point.desc --include_imports point.proto

Then, use the desc file to encode and decode messages.

InputStream input = new FileInputStream("point.desc");
DescriptorProtos.FileDescriptorSet descriptorSet = DescriptorProtos.FileDescriptorSet.parseFrom(input);
DescriptorProtos.FileDescriptorProto fileDescriptorProto = descriptorSet.getFile(0);

Descriptors.Descriptor messageDescriptor = Descriptors.FileDescriptor
        .buildFrom(fileDescriptorProto, new Descriptors.FileDescriptor[0])
        .findMessageTypeByName("Point");


// Encoding
DynamicMessage message = DynamicMessage
        .newBuilder(messageDescriptor)
        .setField(messageDescriptor.findFieldByNumber(1), 10)
        .setField(messageDescriptor.findFieldByNumber(2), 5)
        .setField(messageDescriptor.findFieldByName("label"), "test label")
        .build();

byte[] encodedBytes = message.toByteArray();


// Decoding
DynamicMessage dynamicMessage = DynamicMessage.parseFrom(messageDescriptor, encodedBytes);

int x = (int) dynamicMessage.getField(messageDescriptor.findFieldByName("x"));
int y = (int) dynamicMessage.getField(messageDescriptor.findFieldByName("y"));
String label = (String) dynamicMessage.getField(messageDescriptor.findFieldByName("label"));