Iterating through JSON array in Shell script

Question

I have a JSON data as follows in data.json file

[
  {"original_name":"pdf_convert","changed_name":"pdf_convert_1"},
  {"original_name":"video_encode","changed_name":"video_encode_1"},
  {"original_name":"video_transcode","changed_name":"video_transcode_1"}
]

I want to iterate through the array and extract the value for each element in a loop. I saw jq. I find it difficult to use it to iterate. How can I do that?

Answer 1

Just use a filter that would return each item in the array. Then loop over the results, just make sure you use the compact output option (-c) so each result is put on a single line and is treated as one item in the loop.

jq -c '.[]' input.json | while read i; do
    # do stuff with $i
done

Answer 2

jq has a shell formatting option: @sh.

You can use the following to format your json data as shell parameters:

cat data.json | jq '. | map([.original_name, .changed_name])' | jq @sh

The output will look like:

"'pdf_convert' 'pdf_convert_1'"
"'video_encode' 'video_encode_1'",
"'video_transcode' 'video_transcode_1'"

To process each row, we need to do a couple of things:

• Set the bash for-loop to read the entire row, rather than stopping at the first space (default behavior).
• Strip the enclosing double-quotes off of each row, so each value can be passed as a parameter to the function which processes each row.

To read the entire row on each iteration of the bash for-loop, set the IFS variable, as described in this answer.

To strip off the double-quotes, we'll run it through the bash shell interpreter using xargs:

stripped=$(echo $original | xargs echo)

---

Putting it all together, we have:

#!/bin/bash

function processRow() {
  original_name=$1
  changed_name=$2

  # TODO
}

IFS=$'\n' # Each iteration of the for loop should read until we find an end-of-line
for row in $(cat data.json | jq '. | map([.original_name, .changed_name])' | jq @sh)
do
  # Run the row through the shell interpreter to remove enclosing double-quotes
  stripped=$(echo $row | xargs echo)

  # Call our function to process the row
  # eval must be used to interpret the spaces in $stripped as separating arguments
  eval processRow $stripped
done
unset IFS # Return IFS to its original value

Answer 3

By leveraging the power of Bash arrays, you can do something like:

# read each item in the JSON array to an item in the Bash array
readarray -t my_array < <(jq -c '.[]' input.json)

# iterate through the Bash array
for item in "${my_array[@]}"; do
  original_name=$(jq '.original_name' <<< "$item")
  changed_name=$(jq '.changed_name' <<< "$item")
  # do your stuff
done