NullPointerException in Avro ReflectDatumWriter

Question

I have a specific issue with Avro serialization of Java Objects. I have POJO's generated from xsd schemas which I am then trying to serialize using avro to place on a kafka topic. Some of the xmlElements are optional

A test message looks like this:

@XmlRootElement(name = "message")
public class Testmessage {

  @XmlElement
  public String id;


  @XmlElement
  public String name;

  public Testmessage(String id, String name) {
    this.id = id;
    this.name = name;
  }

  public Testmessage() { }

  @Override
  public String toString() {
    return "Message{" +
        "id='" + id + '\'' +
        ", name=" + name +
        '}';
  }
}

And the method to serialize and place on the topic is:

public void sendMessage(Testmessage msg) throws Exception{

    DatumWriter<Testmessage> writer = new ReflectDatumWriter<Testmessage>(Testmessage.class); 
    ByteArrayOutputStream os = new ByteArrayOutputStream();

    Encoder encoder = EncoderFactory.get().binaryEncoder(os, null);
    writer.write(msg, encoder);
    encoder.flush();
    os.close();
    KeyedMessage<String, byte[]> data = new KeyedMessage<String, byte[]>(TOPIC_NAME, os.toByteArray());

    producer.send(data);

}

When I send both fields all works as expected. If I null one of the fields or leave it out I get NPE's from the write.

java.lang.NullPointerException: in Testmessage in string null of string in field id of    Testmessage
at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:145)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)

Any ideas? or point me in the right direction

Thanks!

Answer 1

It would appear that I ended managing to solve it myself after posting, a couple days of reading the internets later

    ReflectData reflectData = ReflectData.AllowNull.get();
    Schema schema = reflectData.getSchema(Testmessage.class);

    DatumWriter<Testmessage> writer = new ReflectDatumWriter<Testmessage>(schema); 

Appears to allow the use of nulls quite happily.

On to my next error! which is

org.apache.avro.UnresolvedUnionException: Not in union ["null",{"type":"record","name":"XMLGregorianCalendar","namespace":"javax.xml.datatype","fields":[]}]: 2014-10-22
at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:604)
at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:151)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:143)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)

Answer 2

Avro does not support all the data-types. Date and time support has been added very recently too (https://issues.apache.org/jira/browse/AVRO-739)

So this might be due to XMLGregorianCalendar not being supported as a data-type in Avro. Could you try using some other data-type (recommend using a String at first, before moving on to a more complex data type)?