I am testing a new schema registry which loads and retrieves different kinds of avro schemas. In the process of testing, I need to create a bunch of different types of avro schemas. As it involves a lot of permutations, I decided to create the schema programmatically. I am using the apache avro SchemaBuilder to do so.
I created the avro using :
Schema oldSchema = SchemaBuilder
.record("abc")
.aliases("records")
.fields()
.name("field_null")
.type("null")
.noDefault()
.endRecord();
This worked. The avro created looks like :
{
"type" : "record",
"name" : "abc",
"fields" : [ {
"name" : "field_null",
"type" : "null"
} ],
"aliases" : [ "records" ]
}
Now I want to create a new version of the schema using the apache avro libraries like :
{
"type" : "record",
"name" : "abc",
"fields" : [ {
"name" : "field_null",
"type" : "null"
},
{
"name" : "new_field",
"type" : "int",
"default" : 10
}
],
"aliases" : [ "records" ]
}
For this, I tried :
Schema.Field field = new Schema.Field("new_field", SchemaBuilder.builder().intType(),
"NewField", 10);
List<Schema.Field> fields = new ArrayList<>();
fields.add(field);
fields.addAll(oldSchema.getFields());
Schema record = Schema.createRecord(oldSchema.getName(),
"Changes",
oldSchema.getNamespace(),
false,
fields);
I get :
org.apache.avro.AvroRuntimeException: Field already used: field_null type:NULL pos:0
at org.apache.avro.Schema$RecordSchema.setFields(Schema.java:647)
at org.apache.avro.Schema$RecordSchema.<init>(Schema.java:618)
at org.apache.avro.Schema.createRecord(Schema.java:167)
My problem is :
Logical types specify a way of representing a high-level type as a base Avro type. For example, a date is specified as the number of days after the unix epoch (or before using a negative value). This enables extensions to Avro's type system without breaking binary compatibility.
Creating Avro Schemastype − This field comes under the document as well as the under the field named fields. In case of document, it shows the type of the document, generally a record because there are multiple fields. When it is field, the type describes data type.
You can try this to create fields, maybe it's clumsy :
Schema.Field field = new Schema.Field("new_field",SchemaBuilder.builder().intType(),
"NewField", 10);
List<Schema.Field> fields = new ArrayList<>();
for (Schema.Field f : oldSchema.getFields()) {
Schema.Field _field = new Schema.Field(f.name(), f.schema(), f.doc(), f.defaultValue());
fields.add(_field);
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With