sed substitute variable contains newline (preserve it)

Question

I have a multi-line string, downloaded from the Web:

toast the lemonade
blend with the lemonade
add one tablespoon of the lemonade
grill the spring onions
add the lemonade
add the raisins to the saucepan
rinse the horseradish sauce

I have assigned this to $INPUT, like this:

INPUT=$(lynx --dump 'http://example.net/recipes' \
     | python -m json.tool \
     | awk '/steps/,/]/' \
     | egrep -v "steps|]" \
     | sed 's/[",]\|^ *//g; $d')

At this point, $INPUT is ready for substitution into my target file as follows:

sed -i "0,/OLDINPUT/s//$INPUT/" /home/test_file

Of course, sed complains about an unterminated s command - herein lies the problem.

The current workaround I am using is to echo $INPUT prior to giving it to sed, but then the newlines are not preserved. echo strips newlines - which is the problem.

The correct output should maintain its newlines. How can sed be instructed to preserve the newlines?

Answer 1

The hacky direct answer is to replace all newlines with \n, which you can do by adding

| sed ':a $!{N; ba}; s/\n/\\n/g'

to the long command above. A better answer, because substituting shell variables into code is always a bad idea and with sed you wouldn't have a choice, is to use awk instead:

awk -i inplace -v input="$INPUT" 'NR == 1, /OLDINPUT/ { sub(/OLDINPUT/, input) } 1' /home/test_file

This requires GNU awk 4.1.0 or later for the -i inplace.