How best to specify a Protobuf for use with Netty (preferably using the built-in protobuf support)

Question

I'm specifying a protocol in protocol buffers. The transport layer is harnessing Netty's Protocol Buffers support - the significance being that Netty's ProtobufDecoder accepts one, and only one, type of MessageLite.

Now, I want to send a variety of different message types down this channel, each subtype having structured information associated with it. Protocol-buffers doesn't have an inheritance mechanism, so I'm using a kind of composition. I'm not sure if I am going about it the correct way.

My approach has been to categorise my different events with an enum, and encapsulate their differences using optional members. See my .proto below, I've simplified it for the sake of clarity.

My issue here is that the receiving code needs to make the association between EventType.ERROR and ErrorEventDetail. This just feels a little clumsy.

Simplified Events.proto:

package events;

option java_package = "com.example";
option java_outer_classname = "EventProtocol";

message Event {
  enum EventType {
    START = 0;
    DELEGATE = 1;
    ERROR = 2;
    STOP = 3;
  }
  required events.Event.EventType event_type = 1 [default = START];
  required int32 id = 2;
  required int64 when = 3;
  optional StartEventDetail start_event_detail = 4;
  optional DelegateEventDetail delegate_event_detail = 5;
  optional ErrorEventDetail error_event_detail = 6;
  optional StopEventDetail stop_event_detail = 7;
}

message StartEventDetail {
    required string object_name = 1;
}

message DelegateEventDetail {
    required int32 object_id = 2;
    required string task = 3;
}

message ErrorEventDetail {
  required string text = 1;
  required int32 error_code = 2;
  optional Event cause = 3;
}

message StopEventDetail {
    required int32 object_id = 2;
}

Is this optimal? Would I be better off using extends somehow, or perhaps some other use of enum?

Or even, should I be creating a whole new OneToOneDecoder which can identify a message type by some kind of header? I could do this, but I'd rather not...

Thanks

Answer 1

Seems like you are pretty close / already using one of the Google's protobufs techniques which called Union Types

The gist is you have a dedicated type field, that you would "switch" on to know which message to get:

message OneMessage {
  enum Type { FOO = 1; BAR = 2; BAZ = 3; }

  // Identifies which field is filled in.
  required Type type = 1;

  // One of the following will be filled in.
  optional Foo foo = 2;
  optional Bar bar = 3;
  optional Baz baz = 4;
}

where Foo, Bar and Baz are/could be defined in other files as separate messages. And you can switch on the type to get the actual payload (it's Scala, but you can do the same thing with Java's switch):

OneMessage.getType match { 

  case OneMessage.Type.FOO => 

    val foo = OneMessage.getFoo
    // do the processing
    true

  case OneMessage.Type.BAR => 

    val bar = OneMessage.getBar
    // do the processing
    true

  case OneMessage.Type.BAZ => 

    val baz = OneMessage.getBaz
    // do the processing
    true

}

Answer 2

I originally solved the same problem using the extension mechanism, which I document here

But I found the code in Java required to deal with extensions was horribly ugly and verbose, so I switched to the Union method as described. The code is much cleaner as the generated Java code provides a way to get and build each message in one go.

I use two mechanisms for deciding which optional message to extract. I use the switch method also described in another Answer when performance is needed and I use a reflection method when performance is not an issue and I don't want to have to maintain a switch statement, I just create a handle(Message) for each message. An example of the reflection method is given below, in my case the java wrapper is a class called Commands, and is decoded by Netty for me. It first tries to find a handler that has the specific message as a parameter then if that fails it calls a method using the camel case name. For this to work the Enum must be the underscore name of the camel case message.

// Helper that stops me having to create a switch statement for every command
// Relies on the Cmd enum naming being uppercase version of the sub message field names
// Will call the appropriate handle(Message) method by reflection
// If it is a command with no arguments, therefore no sub message it
// constructs the method name from the camelcase of the command enum
private MessageLite invokeHandler(Commands.Command cmd) throws Exception {
    Commands.Command.Cmd com= cmd.getCmd();
    //String name= CaseFormat.UPPER_UNDERSCORE.to(CaseFormat.LOWER_UNDERSCORE, com.name());
    String name= com.name().toLowerCase();
    jlog.debug("invokeHandler() - Looking up {} from {}", name, com.name());
    FieldDescriptor field= Commands.Command.getDescriptor().findFieldByName(name);
    if(field != null) {
        // if we have a matching field then extract it and call the handle method with that as a parameter
        Object c = cmd.getField(field);
        jlog.debug("invokeHandler() - {}\n{}", c.getClass().getCanonicalName(), c);
        Method m = getClass().getDeclaredMethod("handle", String.class, c.getClass());
        return (MessageLite) m.invoke(this, cmd.getUser(), c);
    }
    // else we call a method with the camelcase name of the Cmd, this is for commands that take no arguments other than the user
    String methodName= "handle"+CaseFormat.UPPER_UNDERSCORE.to(CaseFormat.UPPER_CAMEL, com.name());
    jlog.debug("invokeHandler() - using method: {}", methodName);
    Method m = getClass().getDeclaredMethod(methodName, String.class);
    return (MessageLite) m.invoke(this, cmd.getUser());
}