Got a fairly complicated Avro schema (which I can not modify).
Trying to mock JSON example in java:
GenericRecord genericRecord = AvroUtil.jsonToGenericRecord(jsonData, avroSchema);
It keeps failing:
Exception in thread "main" org.apache.avro.AvroTypeException: Expected start-union. Got VALUE_STRING
Is there e.g. online tool that will provide example of JSON data for any given Avro schema? (so that it can match correctly)
Tried mocking JSON data for hours and still no success..
You can use either ConvertRecord or ConvertAvroToJSON to convert your incoming Avro data to JSON. If the incoming Avro files do not have a schema embedded in them, then you will have to provide it, either to an AvroReader (for ConvertRecord) or the "Avro schema" property (for ConvertAvroToJSON).
Apache Avro ships with some very advanced and efficient tools for reading and writing binary Avro but their support for JSON to Avro conversion is unfortunately limited and requires wrapping fields with type declarations if you have some optional fields in your schema.
JSON Schema is a specification for JSON based format for defining the structure of JSON data. It was written under IETF draft which expired in 2011. JSON Schema − Describes your existing data format. Clear, human- and machine-readable documentation.
You can create random data using trevni dependency and test scope. Here you have a sample code
import org.apache.avro.Schema;
import org.apache.trevni.avro.RandomData;
import java.util.Iterator;
public class JSONExample {
public static void main(String [] args){
Schema schema = new Schema.Parser().parse("{\n" +
" \"type\": \"record\",\n" +
" \"namespace\": \"com.acme\",\n" +
" \"name\": \"Test\",\n" +
" \"fields\": [\n" +
" { \"name\": \"name\", \"type\": \"string\" },\n" +
" { \"name\": \"age\", \"type\": \"int\" },\n" +
" { \"name\": \"sex\", \"type\": \"string\" },\n" +
" { \"name\": \"active\", \"type\": \"boolean\" }\n" +
" ]\n" +
"}");
Iterator<Object> it = new RandomData(schema, 1).iterator();
System.out.println(it.next());
}
}
output
{"name": "cjnyvbmetf", "age": -1757126879, "sex": "", "active": false}
maven dependencies
<dependencies>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.8.2</version>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>trevni-core</artifactId>
<classifier>tests</classifier>
<version>1.8.2</version>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>trevni-avro</artifactId>
<classifier>tests</classifier>
<version>1.8.2</version>
</dependency>
</dependencies>
You need to import below packages
compile "org.apache.avro:avro:1.8.2"
compile "org.apache.avro:avro-tools:1.8.2"
compile "org.apache.avro:trevni-core:1.8.2"
compile "org.apache.avro:trevni-avro:1.8.2"
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With