Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache AVRO with Rest

I am evaluating using Apache AVRO for my Jersey REST services. I am using Springboot with Jersey REST.

Currently I am accepting JSON as input which are converted to Java Pojos using the Jackson object mapper.

I have looked in different places but I cannot find any example that is using Apache AVRO with a Jersey end point.

I have found this Github repository (https://github.com/FasterXML/jackson-dataformats-binary/) which has Apache AVRO plugin.

I still cannot find any good example as how to integrate this. Has anyone used Apache AVRO with Jersey? If yes, is there any example I can use?

like image 520
Makky Avatar asked Aug 26 '17 18:08

Makky


People also ask

Is Avro more efficient than JSON?

We think Avro is the best choice for a number of reasons: It has a direct mapping to and from JSON. It has a very compact format. The bulk of JSON, repeating every field name with every single record, is what makes JSON inefficient for high-volume usage.

When should I use Apache Avro?

Apache Avro is especially useful while dealing with big data. It offers data serialization in binary as well as JSON format which can be used as per the use case. The Avro serialization process is faster, and it's space efficient as well.

Does Avro support serialization?

Apache Avro™ is the leading serialization format for record data, and first choice for streaming data pipelines. It offers excellent schema evolution, and has implementations for the JVM (Java, Kotlin, Scala, …), Python, C/C++/C#, PHP, Ruby, Rust, JavaScript, and even Perl.

Does Avro use JSON?

Avro is a row-oriented remote procedure call and data serialization framework developed within Apache's Hadoop project. It uses JSON for defining data types and protocols, and serializes data in a compact binary format.


1 Answers

To start , two things need to happen:

  1. You need to develop a custom ObjectMapper after the fashion of the Avro schema format
  2. You need to supply that custom ObjectMapper to Jersey.

That should look something like this:

@Provider
public class AvroMapperProvider implements ContextResolver<ObjectMapper> {

    final AvroMapper avroMapper = new AvroMapper();

    @Override
    public ObjectMapper getContext(Class<?> type) {
        return avroMapper;
    }
}

Configure your application to use Jackson as the message handler:

public class MyApplication extends ResourceConfig {
    public MyApplication() {
         super(JacksonFeature.class,AvroMapperProvider.class);
    }
}

Alternatively, you can implement a custom MessageBodyReader and MessageBodyWriter that allows you to directly process the payloads on the way in and out:

public class AvroMessageReader implements MessageBodyReader<Person> {

    AvroSchema schema;

    final AvroMapper avroMapper = new AvroMapper();

    public AvroMessageReader(){
        schema = avroMapper.schemaFor(Person.class); //generates an Avro schema from the POJO class.
    }

    @Override
    public boolean isReadable(Class<?> type, Type type1, Annotation[] antns, MediaType mt) {
        return type == Person.class; //determines that this reader can handle the Person class.
    }

    @Override
    public Person readFrom(Class<Person> type, Type type1, Annotation[] antns, MediaType mt, MultivaluedMap<String, String> mm, InputStream in) throws IOException, WebApplicationException {
        return avroMapper.reader(schema).readValue(in);
    }

}

Here, we generate an avro schema from a hypothetical Person class. The JAX-RS runtime will select this reader based on the response from isReadable.

You can then inject the MessageBodyWorkers component into your service implementation class:

@Path("app")
public static class BodyReaderTest{

    @Context
    private MessageBodyWorkers workers;

    @POST
    @Produces("avro/binary")
    @Consumes("avro/binary")
    public String processMessage() {

        workers.getMessageBodyReader(Person.class, Person.class, new Annotation[]{}, MediaType.APPLICATION_JSON_TYPE);
    }
 }

To answer your last comment: Setting the mime type on your handler to the recommended avro/binary ought to do it.

like image 196
kolossus Avatar answered Oct 13 '22 00:10

kolossus