Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to create a valid JSON example from Avro Schema?

Got a fairly complicated Avro schema (which I can not modify).

Trying to mock JSON example in java:

GenericRecord genericRecord = AvroUtil.jsonToGenericRecord(jsonData, avroSchema);

It keeps failing:

Exception in thread "main" org.apache.avro.AvroTypeException: Expected start-union. Got VALUE_STRING

Is there e.g. online tool that will provide example of JSON data for any given Avro schema? (so that it can match correctly)

Tried mocking JSON data for hours and still no success..

like image 224
user9750148 Avatar asked May 08 '18 02:05

user9750148


People also ask

How do I convert Avro to JSON in Java?

You can use either ConvertRecord or ConvertAvroToJSON to convert your incoming Avro data to JSON. If the incoming Avro files do not have a schema embedded in them, then you will have to provide it, either to an AvroReader (for ConvertRecord) or the "Avro schema" property (for ConvertAvroToJSON).

Can Avro be read as JSON?

Apache Avro ships with some very advanced and efficient tools for reading and writing binary Avro but their support for JSON to Avro conversion is unfortunately limited and requires wrapping fields with type declarations if you have some optional fields in your schema.

What is JSON Schema with example?

JSON Schema is a specification for JSON based format for defining the structure of JSON data. It was written under IETF draft which expired in 2011. JSON Schema − Describes your existing data format. Clear, human- and machine-readable documentation.


2 Answers

You can create random data using trevni dependency and test scope. Here you have a sample code

import org.apache.avro.Schema;
import org.apache.trevni.avro.RandomData;

import java.util.Iterator;

public class JSONExample {
    public static void main(String [] args){
        Schema schema = new Schema.Parser().parse("{\n" +
                "     \"type\": \"record\",\n" +
                "     \"namespace\": \"com.acme\",\n" +
                "     \"name\": \"Test\",\n" +
                "     \"fields\": [\n" +
                "       { \"name\": \"name\", \"type\": \"string\" },\n" +
                "       { \"name\": \"age\", \"type\": \"int\" },\n" +
                "       { \"name\": \"sex\", \"type\": \"string\" },\n" +
                "       { \"name\": \"active\", \"type\": \"boolean\" }\n" +
                "     ]\n" +
                "}");

        Iterator<Object> it = new RandomData(schema, 1).iterator();
        System.out.println(it.next());
    }
}

output

{"name": "cjnyvbmetf", "age": -1757126879, "sex": "", "active": false}

maven dependencies

<dependencies>
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro</artifactId>
        <version>1.8.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>trevni-core</artifactId>
        <classifier>tests</classifier>
        <version>1.8.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>trevni-avro</artifactId>
        <classifier>tests</classifier>
        <version>1.8.2</version>
    </dependency>
</dependencies>
like image 161
hlagos Avatar answered Oct 21 '22 12:10

hlagos


You need to import below packages

compile "org.apache.avro:avro:1.8.2"
compile "org.apache.avro:avro-tools:1.8.2"
compile "org.apache.avro:trevni-core:1.8.2"
compile "org.apache.avro:trevni-avro:1.8.2"
like image 1
Juvin James Avatar answered Oct 21 '22 11:10

Juvin James