Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Flatten nested JSON using jq

Tags:

People also ask

How do I flatten a nested JSON object?

Flatten a JSON object: var flatten = (function (isArray, wrapped) { return function (table) { return reduce("", {}, table); }; function reduce(path, accumulator, table) { if (isArray(table)) { var length = table.

Should I flatten JSON?

There is no need to "flatten" JSON as described in your link. (In fact, it's somewhat contrary to JSON "philosophy".) Sometimes JSON is poorly constructed, with extra layers of "object" that are unnecessary, but the referenced example is not that case.

What does flatten JSON mean?

Use the Flatten JSON Objects extension to convert a nested data layer object into a new object with only one layer of key/value pairs. For more advanced options in converting a data layer, see the Data Layer Converter.

Can you have nested arrays in JSON?

Attributes and event data can contain nested (JSON) values—arrays, objects, and arrays of objects.


I'd like to flatten a nested json object, e.g. {"a":{"b":1}} to {"a.b":1} in order to digest it in solr.

I have 11 TB of json files which are both nested and contains dots in field names, meaning not elasticsearch (dots) nor solr (nested without the _childDocument_ notation) can digest it as is.

The other solutions would be to replace dots in the field names with underscores and push it to elasticsearch, but I have far better experience with solr therefore I prefer the flatten solution (unless solr can digest those nested jsons as is??).

I will prefer elasticsearch only if the digestion process will take far less time than solr, because my priority is digesting as fast as I can (thus I chose jq instead of scripting it in python).

Kindly help.

EDIT:

I think the pair of examples 3&4 solves this for me: https://lucidworks.com/blog/2014/08/12/indexing-custom-json-data/

I'll try soon.