Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

generic protobuf deserializer in Java

I'm trying to write a generic java class that can be used to deserialize/parse any protobuf message.

Here's how the code should look in a perfect world:

public abstract class ProtoDeserializer<T extends Message> {

    public T deserialize(final byte[] bytes) throws Exception {     
        Parser<T> parser = T.getParserForType(); // Syntax Error: this method is not static!

        T message = parser.parseFrom(bytes);
        validate(message);
        return message;
    }

    public abstract void validate(final T message) throws Exception;
}

However, I am not able to get the correct parser for a generic protobuf message. What is the correct way to implement such a generic class?

like image 745
maja Avatar asked May 23 '17 12:05

maja


People also ask

What is Protobuf in Java?

Protocol Buffers is a library from Google. It provides efficient and language-independent ways to serialize the data. It supports serialization and deserialization from languages like Java, Python, Go, Dart, etc. It is one of the most popular serialization libraries used across industries by various companies.

How do I send Protobuf to Kafka?

Plug the KafkaProtobufSerializer into KafkaProducer to send messages of Protobuf type to Kafka. When providing an instance of a Protobuf generated class to the serializer, the serializer can register the Protobuf schema, and all referenced schemas.

How do I create a .proto file?

You can use any text editor to create a . proto file. If you'd like to have syntax highlighting there are also editors that would give you that. I use IntelliJ but a quick Google search found an Eclipse plugin which appears to be free: https://code.google.com/p/protobuf-dt/.

What is a Protobuf schema?

A protobuf message format is defined in the .proto file. Protobuf is recommended over other data formats when you need language interoperability, faster serialization and deserialization, type safety, schema adherence between data producer and consumer applications, and reduced coding effort.


1 Answers

Simplest is to pass the parser as an argument to the constructor:

public abstract class ProtoDeserializer<T extends Message> {

    private final Parser<T> parser;

    public ProtoDeserializer(Parser<T> parser) {
        this.parser = parser;
    }

    public T deserialize(final byte[] bytes) throws Exception {    
        T message = parser.parseFrom(bytes);
        validate(message);
        return message;
    }

    public abstract void validate(final T message) throws Exception;
}

Passing the parser is my current workaround. But it would be nice to avoid it, because it's redundant information.

It may be redundant to you, but it is not redundant to the compiler/runtime.

If you consider that it is possible to create raw implementation of your class:

ProtoDeserializer proto = new ProtoDeserializer() {
    ...
};

The type T would have to come from somewhere.

This is just the reality of erased generics. If you need type information of a generic parameter, you will have to manually supply it.


Another hack you can try is to get the concrete type parameter from an implementing sub class:

private final Parser<T> parser; 

public ProtoDeserializer() {
    Class<?> subclass = this.getClass();

    try {
        ParameterizedType pType = (ParameterizedType) subclass.getGenericSuperclass();
        Class<T> tClass = (Class<T>) pType.getActualTypeArguments()[0];
        // In the case where the constructor for `T` takes no arguments.
        parser = tClass.newInstance().getParserForType();
    } catch(Throwable t) {
        throw new RuntimeException("Subclass not compatible", t);
    }
}

This would work as long as the subclass directly implements ProtoDeserializer with a concrete type argument. i.e.:

class MyDeserializer extends ProtoDeserializer<MyMessage> {...}
like image 198
Jorn Vernee Avatar answered Sep 23 '22 05:09

Jorn Vernee