Storing JQ NULL-delimited output in bash array

Question

on bash 4.4.12 using jq 1.5 with this one-liner IFS=_ read -r -a a < <(jq -ncj '["a","b","c"][]+"_"') ; printf '%s\n' "${a[@]}" I get a properly delimited output

a

b

c

for elements a, b and c respectively, BUT if I try the same thing with a null delimiter like so: IFS= read -r -a a < <(jq -ncj '["a","b","c"][]+"\u0000"') ; printf '%s\n' "${a[@]}" then I would get only one array element containing

abc

Why doesn't this work like expected?

Furthermore, if you try IFS= read -r -d '' -a a < <(jq -ncj '["a","b","c"][]+"\u0000"') ; printf '%s\n' "${a[@]}, you will be surprised to get an array with only the first "a" element:

a

My goal is to find an approach without iterating over elements with any kind of a loop.

Edit: **readarray -d** is not a solution since i need the piece of code to run in bash prior to version 4.4

Answer 1

Use readarray, which gained a -d analogous to the same option on read in bash 4.4:

$ readarray -d $'\0' -t a < <(jq -ncj '["a","b","c"][]+"\u0000"')
$ declare -p a
declare -a a=([0]="a" [1]="b" [2]="c")

-d '' works as well; since shell strings are null terminated, '' is, technically, the string containing the null character.

---

Without readarray -d support, you can use a while loop with read, which should work in any version of bash:

a=()
while read -d '' -r item; do
    a+=("$item")
done < <( jq -ncj '["a","b","c"][]+"\u0000"' )

This is the best you can do unless you know something about the array elements that would let you pick an alternate delimiter that isn't part of any of the elements.

Answer 2

I'm assuming that you want to switch to using a null delimiter instead of _ in order to increase reliability of your scripts. However, the safest way to read json elements is not by using the null delimiter since that is allowed json text according to RFC7159 (page 8). E.g. if ["a","b","c"] were to look like ["a","b\u0000","c"] and you were to append the null char to each of the strings and parse these with a null delimiter, the "b" element would go into two separate bash array slots.

Instead, given that newlines are always escaped within json-strings when using e.g. jq -c I suggest relying on the part of the spec that says

"A string begins and ends with quotation marks."

With that in mind we can define:

jsonStripQuotes(){ local t0; while read -r t0; do t0="${t0%\"}"; t0="${t0#\"}"; printf '%s\n' "$t0"; done < <(jq '.');}

And then, e.g.

echo '["a\u0000 b\n","b\nnn","c d"]' | jq .[] | jsonStripQuotes

..should safely print each json string on separate lines(expanded newline appended), with all newlines and null's within the strings escaped. After that I would do a read with IFS set to newline only:

while IFS=$'\n' do read -r elem; Arr+=("$elem") ; done < <(echo '["a\u0000 b\n","b\nnn","c d"]' | jq .[] | stripJsonQuotes)

And then if you want to print them with newlines etc. expanded:

printf '%b' "${Arr[*]}"

I believe this is the most reliable way to parse json strings to a bash array.