Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I validate JSON against Avro schema

I have a JSON response from an API and I would like to validate against an existing Avro schema(strict validation with names and types).

The response is of type

{"name":"alex","age":23,"sex":"M","active":"true"}

The schema has the above types with the data types and I would like to validate the schema and throw an exception in case it fails.(Preferably JAVA).

I have read a solution using the command line but I wanted to do it programmatically.

Thanks in advance

like image 221
Bhargava Sharma Avatar asked May 07 '18 17:05

Bhargava Sharma


People also ask

How do you validate a JSON file against a schema?

The simplest way to check if JSON is valid is to load the JSON into a JObject or JArray and then use the IsValid(JToken, JsonSchema) method with the JSON Schema. To get validation error messages, use the IsValid(JToken, JsonSchema, IList<String> ) or Validate(JToken, JsonSchema, ValidationEventHandler) overloads.

Can we validate JSON with schema?

JSON Schema is a powerful tool. It enables you to validate your JSON structure and make sure it meets the required API. You can create a schema as complex and nested as you need, all you need are the requirements. You can add it to your code as an additional test or in run-time.

How do I validate an Avro file?

avro-tools is external tool that can be used to convert Avro files to JSON/Text or vice-versa. Once data is imported we can copy the files from HDFS to local file system. We can run avro-tools tojson command to convert Avro file into JSON.


1 Answers

This is how you can validate it programatically.

import org.apache.avro.AvroTypeException;
import org.apache.avro.Schema;
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.Decoder;
import org.apache.avro.io.DecoderFactory;

import java.io.*;

public class MainClass {
    public static void main (String [] args) throws Exception {
        Schema schema = new Schema.Parser().parse("{\n" +
                "     \"type\": \"record\",\n" +
                "     \"namespace\": \"com.acme\",\n" +
                "     \"name\": \"Test\",\n" +
                "     \"fields\": [\n" +
                "       { \"name\": \"name\", \"type\": \"string\" },\n" +
                "       { \"name\": \"age\", \"type\": \"int\" },\n" +
                "       { \"name\": \"sex\", \"type\": \"string\" },\n" +
                "       { \"name\": \"active\", \"type\": \"boolean\" }\n" +
                "     ]\n" +
                "}");
        String json = "{\"name\":\"alex\",\"age\":23,\"sex\":\"M\",\"active\":true}";
        System.out.println(validateJson(json, schema));
        String invalidJson = "{\"name\":\"alex\",\"age\":23,\"sex\":\"M\"}"; // missing active field
        System.out.println(validateJson(invalidJson, schema));
    }

    public static boolean validateJson(String json, Schema schema) throws Exception {
        InputStream input = new ByteArrayInputStream(json.getBytes());
        DataInputStream din = new DataInputStream(input);

        try {
            DatumReader reader = new GenericDatumReader(schema);
            Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);
            reader.read(null, decoder);
            return true;
        } catch (AvroTypeException e) {
            System.out.println(e.getMessage());
            return false;
        }
    }
}
like image 97
hlagos Avatar answered Oct 20 '22 23:10

hlagos