Merging JSON files using a bash script

Question

I am very new to bash scripting. I attempted to write a script that merges several json files. For example:

File 1:

{   "file1": {      "foo": "bar"   } } 

File 2:

{   "file1": {      "lorem": "ipsum"   } } 

Merged File:

{   "file1": {     "foo": "bar"   },   "file2": {     "lorem": "ipsum"   } } 

This is what I came up with:

awk 'BEGIN{print "{"} FNR > 1 && last_file == FILENAME {print line} FNR == 1 {line = ""} FNR==1 && FNR != NR {printf ","} FNR > 1 {line = $0} {last_file = FILENAME} END{print "}"}' json_files/* > json_files/all_merged.json 

It works but I feel there is a better way of doing this. Any ideas?

Answer 1

Handling JSON with awk is not a terribly good idea. Arbitrary changes in meaningless whitespace will break your code. Instead, use jq; it is made for this sort of thing. To combine two objects, use the * operator, i.e., for two files:

jq -s '.[0] * .[1]' file1.json file2.json 

And for arbitrarily many files, use reduce to apply it sequentially to all:

jq -s 'reduce .[] as $item ({}; . * $item)' json_files/* 

The -s switch makes jq read the contents of the JSON files into a large array before handling them.