Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

envsubst: default values for unset variables

I've got a json file input.json like the following one:

{
  "variable" : "${ENV_VAR}"
}

of course, I can invoke envsubst from bash like the following:

$ export ENV_VAR=myvalue
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "myvalue"
}

Now, I wish I could set default values for variables in the input.json for the case when ENV_VAR is not set, like in the following example which, as unfortunately can be seen in the example below, doesn't work:

$ cat input.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}
$ export ENV_VAR=newvalue
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}
$ unset ENV_VAR
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}

What's curious, if I execute the envsubst like in the following example (without involving an input file), it works

$ export ENV_VAR=myvalue
$ echo "value is ${ENV_VAR:=defaultvalue}" | envsubst
value is myvalue
$ unset ENV_VAR
$ echo "value is ${ENV_VAR:=defaultvalue}" | envsubst
value is defaultvalue

Where is the problem with the files?

like image 969
chrx Avatar asked May 08 '18 09:05

chrx


People also ask

What is Strenv?

strenv which also takes a single environment variable name, and always parses the variable as a string. envsubst which you pipe strings into and it interpolates environment variables in strings using envsubst.

What is Envsubst?

The envsubst command searches the input for pattern $VARIABLE or ${VARIABLE}. Then, it replaces the pattern with the value of the corresponding bash variable. In contrast, a pattern that does not refer to any variable is replaced by an empty string. Moreover, envsubst recognizes only exported variables.

What is Shell format?

SHELL-FORMAT is an optional text command line argument containing references to environment variables. To reference an environment variable in the text, prefix the variable name with a $ . For example: Hello $FOO World $BAR references environment variables FOO and BAR . The rest of the string is ignored.


2 Answers

According to man envsubst, envsubst will only ever replace references to environment variables in the form of ${VAR} or $VAR. Special shell features like ${VAR:-default} are not supported. The only thing you could do is to (re)define all variables in the environment of the envsubst invocation and assign local default values, if they are missing:

ENV_VAR="${ENV_VAR:-defaultvalue}" \
OTHER_VAR="${OTHER_VAR:-otherdefault}" \
envsubst < input.json > output.json

Note, that this is actually a single command line split into multiple lines each ending with a line continuation \. The first two lines are variable assignments, that are only effective in the environment of the executed command envsubst in the last line. What's happening is, that the shell will create an environment for the execution of the command (as it would always do). That environment is initially a copy of the current shell environment. Within that new environment ENV_VAR and OTHER_VAR are assigned the values of expanding the expression ${VAR:-default}, which essentially expands to default unless VAR is defined and has a none-empty value. The command envsubst is executed, receiving the file input.json as standard-input and having its standard-output redirected to output.json (both is done by the shell, transparent to the command). After the command execution, the shell deletes the command environment returning to its original environment, i.e. the local variable assignments are no longer effective.

There is no way to define default values from inside the JSON file, unless you implement a program to do so yourself, or use another tool that can to that.

You could do something like the following, but it is NOT RECOMMENDED:

eval echo "$(cat input.json)" > output.json

which will read input.json into a string, and than evaluate the command echo <string> as if it was type literally, which means that any embedded ${VAR:-default} stuff should be expanded by the shell before the string is passed to echo. BUT any other embedded shell feature will be evaluated as well, which poses a HUGE SECURITY RISK.

like image 151
Robin479 Avatar answered Sep 29 '22 01:09

Robin479


I'm using https://github.com/a8m/envsubst and it has enhancements over the original gettext envsubst that the expressions in the template file supports default values.

The example in the README just works.

echo 'welcome $HOME ${USER:=a8m}' | envsubst
like image 29
mash Avatar answered Sep 29 '22 01:09

mash