Bash Templating: How to build configuration files from templates with Bash?

Question

Try envsubst

FOO=foo
BAR=bar
export FOO BAR

envsubst <<EOF
FOO is $FOO
BAR is $BAR
EOF

You can use this:

perl -p -i -e 's/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg' < template.txt

to replace all ${...} strings with corresponding enviroment variables (do not forget to export them before running this script).

For pure bash this should work (assuming that variables do not contain ${...} strings):

#!/bin/bash
while read -r line ; do
    while [[ "$line" =~ (\$\{[a-zA-Z_][a-zA-Z_0-9]*\}) ]] ; do
        LHS=${BASH_REMATCH[1]}
        RHS="$(eval echo "\"$LHS\"")"
        line=${line//$LHS/$RHS}
    done
    echo "$line"
done

. Solution that does not hang if RHS references some variable that references itself:

#!/bin/bash
line="$(cat; echo -n a)"
end_offset=${#line}
while [[ "${line:0:$end_offset}" =~ (.*)(\$\{([a-zA-Z_][a-zA-Z_0-9]*)\})(.*) ]] ; do
    PRE="${BASH_REMATCH[1]}"
    POST="${BASH_REMATCH[4]}${line:$end_offset:${#line}}"
    VARNAME="${BASH_REMATCH[3]}"
    eval 'VARVAL="$'$VARNAME'"'
    line="$PRE$VARVAL$POST"
    end_offset=${#PRE}
done
echo -n "${line:0:-1}"

WARNING: I do not know a way to correctly handle input with NULs in bash or preserve the amount of trailing newlines. Last variant is presented as it is because shells “love” binary input:

1. read will interpret backslashes.
2. read -r will not interpret backslashes, but still will drop the last line if it does not end with a newline.
3. "$(…)" will strip as many trailing newlines as there are present, so I end … with ; echo -n a and use echo -n "${line:0:-1}": this drops the last character (which is a) and preserves as many trailing newlines as there was in the input (including no).

envsubst was new to me. Fantastic.

For the record, using a heredoc is a great way to template a conf file.

STATUS_URI="/hows-it-goin";  MONITOR_IP="10.10.2.15";

cat >/etc/apache2/conf.d/mod_status.conf <<EOF
<Location ${STATUS_URI}>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from ${MONITOR_IP}
</Location>
EOF

I agree with using sed: it is the best tool for search/replace. Here is my approach:

$ cat template.txt
the number is ${i}
the dog's name is ${name}

$ cat replace.sed
s/${i}/5/
s/${name}/Fido/

$ sed -f replace.sed template.txt > out.txt

$ cat out.txt
the number is 5
the dog's name is Fido

I have a bash solution like mogsie but with heredoc instead of herestring to allow you to avoid escaping double quotes

eval "cat <<EOF
$(<template.txt)
EOF
" 2> /dev/null

I think eval works really well. It handles templates with linebreaks, whitespace, and all sorts of bash stuff. If you have full control over the templates themselves of course:

$ cat template.txt
variable1 = ${variable1}
variable2 = $variable2
my-ip = \"$(curl -s ifconfig.me)\"

$ echo $variable1
AAA
$ echo $variable2
BBB
$ eval "echo \"$(<template.txt)\"" 2> /dev/null
variable1 = AAA
variable2 = BBB
my-ip = "11.22.33.44"

This method should be used with care, of course, since eval can execute arbitrary code. Running this as root is pretty much out of the question. Quotes in the template need to be escaped, otherwise they will be eaten by eval.

You can also use here documents if you prefer cat to echo

$ eval "cat <<< \"$(<template.txt)\"" 2> /dev/null

@plockc provoded a solution that avoids the bash quote escaping issue:

$ eval "cat <<EOF
$(<template.txt)
EOF
" 2> /dev/null

Edit: Removed part about running this as root using sudo...

Edit: Added comment about how quotes need to be escaped, added plockc's solution to the mix!