Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Shell script to join 5 or more json files together

I am working on a project that has a lot of json documents in a large file lets call it manifest.json

The files have titles such as

a-11.json

{"id":"a-11",
 "name":"XN0",
  "code":"H3A8FF82820F"
  "status":"live"
} 

a-03.json

{"id":"a-03",
 "name":"PF1",
  "code":"FFFF82820F"
  "status":"live"
}

a-09.json

{"id":"a-09",
 "name":"PF1",
 "code":"FFFF82820F"
 "status":"live"
} 

What I want a shell script to do is to concatenate them all in alpha order and I am also need to wrap them like this: [ {json doc}, {json doc}, {json doc ] with a sq bracket seperated with a , so that it looks like the code below-

The join command only connects two files so thats not going to work and I have tried a combination of cat and ls but it all goes a bit wrong. I am trying to use the Linux environment rather than the MS environment here.

manifest.json

[
{"id":"a-03",
 "name":"PF1",
  "code":"FFFF82820F"
  "status":"live"
},
{"id":"a-09",
 "name":"PF1",
  "code":"FFFF82820F"
  "status":"live"
}, 
{"id":"a-11",
 "name":"XN0",
  "code":"H3A8FF82820F"
  "status":"live"
}

]

The command

cat a-*.json > manifest.json

Gives me the following with the a-11.json doc at the top, any help appreciated.

[
{"id":"a-11",
 "name":"XN0",
  "code":"H3A8FF82820F"
  "status":"live"
}
{"id":"a-03",
 "name":"PF1",
  "code":"FFFF82820F"
  "status":"live"
},
{"id":"a-09",
 "name":"PF1",
  "code":"FFFF82820F"
  "status":"live"
}, 

]
like image 943
Richard Avatar asked Apr 30 '14 11:04

Richard


People also ask

Can we merge two json files?

Merge two JSON files without using a third file in Python You can do it by importing json library but it would be a little complex for beginners who have no idea about json objects and Python dictionary. So, here we will do it using basic File Handling in Python as it will be much easier for you!


2 Answers

You could use jq utility (sed for JSON data):

$ jq -s '.' a-*.json > manifest.json

manifest.json:

[
  {
    "status": "live",
    "code": "H3A8FF82820F",
    "name": "XN0",
    "id": "a-11"
  },
  {
    "status": "live",
    "code": "FFFF82820F",
    "name": "PF1",
    "id": "a-03"
  },
  {
    "status": "live",
    "code": "FFFF82820F",
    "name": "PF1",
    "id": "a-09"
  }
]
like image 182
jfs Avatar answered Oct 10 '22 15:10

jfs


Use the following bash script (it uses arrays which aren't standard across all shells):

#!/bin/bash

shopt -s nullglob
declare -a jsons
jsons=(a-*.json) # ${jsons[@]} now contains the list of files to concatenate
echo '[' > manifest.json
if [ ${#jsons[@]} -gt 0 ]; then # if the list is not empty
  cat "${jsons[0]}" >> manifest.json # concatenate the first file to the manifest...
  unset jsons[0]                     # and remove it from the list
  for f in "${jsons[@]}"; do         # iterate over the rest
      echo "," >>manifest.json
      cat "$f" >>manifest.json
  done
fi
echo ']' >>manifest.json             # complete the manifest
like image 41
user3159253 Avatar answered Oct 10 '22 16:10

user3159253