Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Map one JSON schema to a different JSON schema

Tags:

json

json-ld

I have a bunch of JSON files, thousands of different schemas. Using GenSON (the Python JSON schema generator), I managed to create schema files for each of the input files. Now, what I'd like to do is standardize all these different files to one defined schema. Here's an example:

Input

{
     "name": "Bob Odenkirk",
     "title": "Software Engineer",
     "location": {
         "locality": "San Francisco",
         "region": "CA",
         "country": "United States"
     },
     "age": 62,
     "status": "Active"
}

Output

{
     "names": ["Bob Odenkirk"],
     "occupations": ["Software Engineer"],
     "locations": ["San Francisco, CA"]    
}

Essentially, I am looking for a language agnostic method (i.e., I don't care what programming language is used) of defining how an input JSON file should be parsed to an output JSON file.

like image 453
Franz Kafka Avatar asked Nov 25 '17 15:11

Franz Kafka


People also ask

How do I Map one JSON to another JSON?

You can use the Jackson API to add field or transform a JSON without creating POJO's. It provides a object form called JsonNode , JsonObject and JsonArray types which can be transformed like i did in the below code. I hope this helps you.

How do you reference a JSON Schema in another JSON Schema?

A schema can reference another schema using the $ref keyword. The value of $ref is a URI-reference that is resolved against the schema's Base URI. When evaluating a $ref , an implementation uses the resolved identifier to retrieve the referenced schema and applies that schema to the instance.

Can a JSON file reference another JSON file?

JSON Reference allows a JSON value to reference another value in a JSON document. This module implements utilities for exploring these objects.

Can I model a JSON message without a JSON Schema?

If you do not have a JSON schema, you can model a JSON message that contains JSON objects, JSON arrays, or both, by following the steps in this topic to create an equivalent XML schema model, which you can then use in one or more message maps with the Cast function in the Graphical Data Mapping editor.

Can I use JSON Schema in mapping node?

When you cast the JSON.Data.any on the input side of the map to define the JSON message, you can only use schema models that are defined in the default namespace. For example, if you build a JSON input with a cast to an element in a none empty namespace, the Mapping node runs without throwing an error.

What is the current version of JSON Schema?

The current version is 2019-09! JSON Schema is a vocabulary that allows you to annotate and validate JSON documents.

How to convert spark dataframe with JSON string into maptype (Map) column?

By using syntax from_json (Column jsonStringcolumn, DataType schema), you can convert Spark DataFrame with JSON string into MapType (map) column. MapType is a subclass of DataType. import org.apache.spark.sql.functions.{ from_json, col } import org.apache.spark.sql.types.{


3 Answers

The url https://github.com/bazaarvoice/jolt#jolt says that Jolt may be what you're looking for.

Jolt

JSON to JSON transformation library written in Java where the "specification" for the transform is itself a JSON document.

Useful For

Transforming JSON data from ElasticSearch, MongoDb, Cassandra, etc before sending it off to the world

Extracting data from a large JSON documents for your own consumption

like image 163
mikep Avatar answered Nov 03 '22 02:11

mikep


Jolt Spec

[
  // First build the "city, state" string for location
  {
    "operation": "modify-default-beta",
    "spec": {
      "location": {
        "locConcat": "=concat(@(1,locality),', ',@(1,region))"
      }
    }
  },
  // Then map the fields as needed to positions in an output json
  {
    "operation": "shift",
    "spec": {
      "name": "name[0]",
      "title": "occupations[0]",
      "location": {
        "locConcat": "locations[0]"
      }
    }
  }
]
like image 45
Milo S Avatar answered Nov 03 '22 02:11

Milo S


I am not sure is your expecting like below. Long time back I have created flat object and output format object. It will return output format object with data filled.

var input = {
     "name": "Bob Odenkirk",
     "title": "Software Engineer",
     "location": {
         "locality": "San Francisco",
         "region": "CA",
         "country": "United States"
     },
     "age": 62,
     "status": "Active"
};

var outputFormat = {
    "name": "name",
  "occupations": "title",
  "locations": "location.locality, location.region"
};

var flatInput = {};

function generateFlatInput(input, parent){
     for (var prop in input) {
        if(input.hasOwnProperty(prop) && typeof input[prop] === 'object')
            flatInput = generateFlatInput(input[prop], parent + prop + '.');
        else
         flatInput[parent + prop] =  input[prop];
    }

    return flatInput;
}

function generateOutput(input, outputFormat, delimiter){
     input = generateFlatInput(input, '');

     for (var prop in outputFormat) {
        var fields = outputFormat[prop].split(delimiter);
      var fieldValue = [];
      for(i = 0; i < fields.length; i++){
                    if(!input.hasOwnProperty(fields[i].trim())) continue;

          fieldValue.push(input[fields[i].trim()]);
      }
      outputFormat[prop] = fieldValue.join(delimiter);
   }

   return outputFormat;
}

console.log(generateOutput(input, outputFormat, ', '));

https://jsfiddle.net/u2yyuguk/1/

like image 44
Kalaikumar Thangasamy Avatar answered Nov 03 '22 01:11

Kalaikumar Thangasamy