bash script to replace all occurrences of placeholders in file

Question

I'm trying to write a bash script to replace all occurrences of a placeholder in a file with an environment variable of the same name. As an example, if I have a file like the following...

This is an {{VAR1}} {{VAR2}}.
It should work across multiple lines in this {{VAR2}}.

... and I have the following environment variables set:

VAR1='example'
VAR2='file'

after running the script on my file, I should get the output:

This is an example file.
It should work across multiple lines in this file.

I'm sure there must be a solution using awk/sed, but so far the closest I've come can't handle if there's more than one variable on a line. Here's my attempt so far:

cat example.txt | grep -o '{{.*}}' > temp
while read placeholder; do
  varName=$(echo "$placeholder" | tr -d '{}')
  value="${!varName}"
  sed -i "s/$placeholder/$value/g" "$file"
done < temp
rm -rf temp

Answer 1

I'd use Perl:

perl -pe 's/{{(.*?)}}/$ENV{$1}/g' filename

This assumes that VAR1 and VAR2 are environment variables (i.e., are exported), so that Perl can pick them out of its environment. This would be required of any approach that isn't pure shell; I just mention it to avoid confusion.

This works as follows:

• s/pattern/replacement/g is a substitution command; you may recognize it from sed. The difference is that here we can use Perl's more powerful regex engine and variables. The g flag makes it so that all matches are replaced; without it, it would apply only to the first.
• In the pattern, .*? matches non-greedily, so that in a line that contains foo {{VAR1}} bar {{VAR2}} baz, the pattern {{.*?}} matches only {{VAR1}} instead of {{VAR1}} bar {{VAR2}}.
• The part between {{ and }} is captured because it is between () and can be reused as $1
• $ENV{$1} in the replacement uses the special %ENV hash that contains the environment of the Perl process. $ENV{$1} is the value of the environment variable that has the name $1, which is the captured group from before.

Answer 2

Only bash and sed:

$ VAR1='example'
$ VAR2='file'
$ export VAR1 VAR2

$ sed -e '{s/{{\([^{]*\)}}/${\1}/g; s/^/echo "/; s/$/";/}' -e e filename
This is an example file.
It should work across multiple lines in this file.

• sed -e '{s/{{\([^{]*\)}}/${\1}/g;}' filename:

  This is an ${VAR1} ${VAR2}.
  It should work across multiple lines in this ${VAR2}.
  

  • {{\([^{]*\)}} - Search for {{..}}
  • [^{] - Non greedy match
  • \1 - Access to the bracketed values \(...\).

• sed -e '{s/{{\([^{]*\)}}/${\1}/g; s/^/echo "/; s/$/";/}' filename:

  echo "This is an ${VAR1} ${VAR2}.";
  echo "It should work across multiple lines in this ${VAR2}.";
  

  • s/^/echo "/ - Replace the beginning of the line with echo "
  • s/$/";/ - Replace the end of the line with ";