bash trouble assigning to an array index in a loop

Question

I can get this to work in ksh but not in bash which is really driving me nuts. Hopefully it is something obvious that I'm overlooking.

I need to run an external command where each line of the output will be stored at an array index.

This simplified example looks like it is setting the array in the loop correctly however after the loop has completed those array assignments are gone? It's as though the loop is treated completely as an external shell?

junk.txt

this is a
test to see
if this works ok

testa.sh

#!/bin/bash

declare -i i=0
declare -a array

echo "Simple Test:"
array[0]="hello"
echo "array[0] = ${array[0]}"

echo -e "\nLoop through junk.txt:"
cat junk.txt | while read line
do
    array[i]="$line"
    echo "array[$i] = ${array[i]}"
    let i++
done

echo -e "\nResults:"
echo "       array[0] = ${array[0]}"
echo " Total in array = ${#array[*]}"
echo "The whole array:"
echo ${array[@]}

Output

Simple Test:
array[0] = hello

Loop through junk.txt:
array[0] = this is a
array[1] = test to see
array[2] = if this works ok

Results:
      array[0] = hello
Total in array = 1
The whole array:
hello

So while in the loop, we assign array[i] and the echo verifies it. But after the loop I'm back at array[0] containing "hello" with no other elements.

Same results across bash 3, 4 and different platforms.

Answer 1

Because your while loop is in a pipeline, all variable assignments in the loop body are local to the subshell in which the loop is executed. (I believe ksh does not run the command in a subshell, which is why you have the problem in bash.) Do this instead:

while read line
do
    array[i]="$line"
    echo "array[$i] = ${array[i]}"
    let i++
done < junk.txt

Rarely, if ever, do you want to use cat to pipe a single file to another command; use input redirection instead.

UPDATE: since you need to run from a command and not a file, another option (if available) is process substitution:

while read line; do
...
done < <( command args ... )

If process substitution is not available, you'll need to output to a temporary file and redirect input from that file.

If you are using bash 4.2 or later, you can execute these two commands before your loop, and the original pipe-into-the-loop will work, since the while loop is the last command in the pipeline.

set +m    # Turn off job control; it's probably already off in a non-interactive script
shopt -s lastpipe
cat junk.txt | while read line; do ...; done

UPDATE 2: Here is a loop-less solution based on user1596414's comment

array[0]=hello
IFS=$'\n' array+=( $(command) )

The output of your command is split into words based solely on newlines (so that each line is a separate word), and appends the resulting line-per-slot array to the original. This is very nice if you are only using the loop to build the array. It can also probably be modified to accomodate a small amount of per-line processing, vaguely similar to a Python list comprehension.