Reading Avro file gives AvroTypeException: missing required field error (even though the new field is declared null in schema)

Question

I am trying to deserialize/read an Avro file, the avro data file doesn't have the new field. Even though the new field is declared as null in schema, it is expected to be optional. But it still gives me error as mandatory.

Exception in thread "main" org.apache.avro.AvroTypeException: Found com.kiran.avro.User, expecting com.kiran.avro.User, missing required field loc

The AVRO schema declaration:

{"name": "loc", "type": ["string", "null"]}

Reading file using code:

DatumReader<User> userDatumReader = new SpecificDatumReader<User>(User.class);
DataFileReader<User> dataFileReader = new DataFileReader<User>(file, userDatumReader);

Is there any other way to declare an optional field?

Thanks for hints/suggestions !!

Answer 1

What is the content of "file"?

I might be wrong, but if you define a field in schema as {"name": "loc", "type": ["string", "null"]}, you still need to define a loc field, even for null. It should be something like "loc": null in the file.

Try adding "default" to this field declaration:

{"name" : "loc",
"type" :  ["null","string"] ,
"default" : null}

Then it should be possible to omit this field in file.

You can also see this question Avro: deserialize json - schema with optional fields for some additional info and examples.