Combining Google proto buffers with Jersey/JAX-RS

Question

Currently I have a RESTful web service with endpoints that are exposed via Jersey/JAX-RS:

@Path("/widgets")
public class WidgetResource {
    @GET
    List<Widget> getAllWidgets() {
        // gets Widgets somehow
    }

    @POST
    Widget save(Widget w) {
        // Save widget and return it
    }
}

I use Jackson for serializing/deserializing my POJOs into JSON, and my service both responds to and sends back my POJOs as application/json.

I am now looking to possibly use Google protocol buffers (or an equivalent technology) to help compress/optimize the communication between client and service, as JSON/text is pretty bulky/wasteful.

In reality, I have a large backend that consists of a "microservice" architecture; dozens of REST services communicating with each other; this is why I'm looking to optimize the the messages sent backk and forth between all of them.

So I ask: is it possible to still have Jersey/JAX-RS serve up my service endpoints, but to gut out the Jackson/JSON stuff and replace it with Google protocol buffers? If so, what might this code look like?

Answer 1

JAX-RS uses implementations of MessageBodyReader and MessageBodyWriter to serialize/deserialize to and from differen media types. You can read more at JAX-RS Entity Providers. You can write your own to handle the serializion/derialization of your protobuf objects. Then just register the provider(s) with the application, either explicitly or implicitly through discovery.

Example

widgets.proto

package widget;

option java_package = "protobuf.example";
option java_outer_classname = "WidgetsProtoc";

message Widget {
    required string id = 1;
    required string name = 2;
}

message WidgetList {
    repeated Widget widget = 1;
}

When this is compiled, I will be left with a WidgetsProtoc class with static inner Widget and WidgetList classes.

WidgetResource

import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Response;
import protobuf.example.WidgetsProtoc.Widget;
import protobuf.example.WidgetsProtoc.WidgetList;

@Path("/widgets")
public class WidgetResource {
    
    @GET
    @Produces("application/protobuf")
    public Response getAllWidgets() {
        Widget widget1 = 
                Widget.newBuilder().setId("1").setName("widget 1").build();
        Widget widget2 = 
                Widget.newBuilder().setId("2").setName("widget 2").build();
        WidgetList list = WidgetList.newBuilder()
                .addWidget(widget1).addWidget(widget2).build();
        return Response.ok(list).build();
    }
    
    @POST
    @Consumes("application/protobuf")
    public Response postAWidget(Widget widget) {
        StringBuilder builder = new StringBuilder("Saving Widget \n");
        builder.append("ID: ").append(widget.getId()).append("\n");
        builder.append("Name: ").append(widget.getName()).append("\n");
        return Response.created(null).entity(builder.toString()).build();
    }
}

You'll notice the use of the "application/protobuf" media type. This isn't a standard media type, but there is a draft in the working. Also the Guava library has define this media type as MediaType.PROTOBUF, which translates to "application/protobuf", so I chose to stick with that.

MessageBodyReader and MessageBodyWriter all defined in one class. You can choose to do it separately. Makes no difference.

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.lang.annotation.Annotation;
import java.lang.reflect.Type;
import javax.ws.rs.BadRequestException;
import javax.ws.rs.Consumes;
import javax.ws.rs.Produces;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.ext.MessageBodyReader;
import javax.ws.rs.ext.MessageBodyWriter;
import javax.ws.rs.ext.Provider;
import protobuf.example.WidgetsProtoc.Widget;
import protobuf.example.WidgetsProtoc.WidgetList;

@Provider
@Produces("application/protobuf")
@Consumes("application/protobuf")
public class WidgetProtocMessageBodyProvider 
                   implements MessageBodyReader, MessageBodyWriter {

    @Override
    public boolean isReadable(Class type, Type type1, 
            Annotation[] antns, MediaType mt) {
        return Widget.class.isAssignableFrom(type) 
                || WidgetList.class.isAssignableFrom(type);
    }

    @Override
    public Object readFrom(Class type, Type type1, Annotation[] antns, 
            MediaType mt, MultivaluedMap mm, InputStream in) 
            throws IOException, WebApplicationException {
        if (Widget.class.isAssignableFrom(type)) {
            return Widget.parseFrom(in);
        } else if (WidgetList.class.isAssignableFrom(type)) {
            return WidgetList.parseFrom(in);
        } else {
            throw new BadRequestException("Can't Deserailize");
        }
    }

    @Override
    public boolean isWriteable(Class type, Type type1, 
            Annotation[] antns, MediaType mt) {
        return Widget.class.isAssignableFrom(type) 
                || WidgetList.class.isAssignableFrom(type);
    }

    @Override
    public long getSize(Object t, Class type, Type type1, 
            Annotation[] antns, MediaType mt) {  return -1; }

    @Override
    public void writeTo(Object t, Class type, Type type1, 
            Annotation[] antns, MediaType mt, 
            MultivaluedMap mm, OutputStream out) 
            throws IOException, WebApplicationException {
        if (t instanceof Widget) {
            Widget widget = (Widget)t;
            widget.writeTo(out);
        } else if (t instanceof WidgetList) {
            WidgetList list = (WidgetList)t;
            list.writeTo(out);
        }
    }  
}

TestCase (Make sure the provider is registered both with the server and client)

@Test
public void testGetIt() {
    Client client = ClientBuilder.newClient();
    WebTarget target = client.target("http://localhost:8080/api");
    // Get all list
    WidgetList list = target.path("/widgets").request().get(WidgetList.class);
    System.out.println("===== Response from GET =====");
    for (Widget widget: list.getWidgetList()) {
        System.out.println("id: " + widget.getId() 
                         + ", name: " + widget.getName());
    }
    
    // Post one 
    Widget widget = Widget.newBuilder().setId("10")
                          .setName("widget 10").build();
    Response responseFromPost = target.path("widgets").request()
            .post(Entity.entity(widget, "application/protobuf"));
    System.out.println("===== Response from POST =====");
    System.out.println(responseFromPost.readEntity(String.class));
    responseFromPost.close();
}

Result:

===== Response from GET =====
id: 1, name: widget 1
id: 2, name: widget 2
===== Response from POST =====
Saving Widget 
ID: 10
Name: widget 10