Read line by line from a variable in shell scripting

Question

I have a script variable which is multi-line. How do i traverse this variable to read it line by line and process each line the way I want?

Answer 1

Consider the following multi-line variable

x=$(echo -e "a\nb\nc d e")

and a simple process for each line: just echo it with a prefix=LINE: and with single quotes around the line. Either of the following codes will satisfy that requirement:

while read line; do echo "LINE: '${line}'"; done <<< "$x"

or

while read line; do echo "LINE: '${line}'"; done < <(echo "$x")

Neither creates a subshell (so you can, e.g., set variables in the loop and access them outside of it), and both output

LINE: 'a'
LINE: 'b'
LINE: 'c d e'

But suppose instead you have

x=$(echo -e "a \n b\nc d e")
# note--------^--^

and that leading and trailing whitespace matter for your application (e.g., parsing Git porcelain). Both the above codes will give exactly the same output for the latter variable/data as for the former, which is not what you want. To preserve leading and trailing whitespace, replace while read line with while IFS= read -r line . I.e., either of the following codes

while IFS= read -r line; do echo "LINE: '${line}'"; done <<< "$x"

or

while IFS= read -r line; do echo "LINE: '${line}'"; done < <(echo "$x")

will produce

LINE: 'a '
LINE: ' b'
LINE: 'c d e'

See Greg Wooledge's excellent Bash FAQ for details.

Answer 2

Although I typically use "while read" for processing multi-line variables, I recently had an instance where it removed the leading space from each line in a file. Using this instead fixed my issue:

printf '%s\n' "$var" | while IFS= read -r line
do
   echo "$line"
done

Code taken from this Unix Stack Exchange answer.

Edit: updated to fix last line issue as suggested by Nicolae Iotu