How do I locally source environment variables that I have defined in a Docker-format env-file?

Question

I've written a bunch of environment variables in Docker format, but now I want to use them outside of that context. How can I source them with one line of bash?

---

Details

Docker run and compose have a convenient facility for importing a set of environment variables from a file. That file has a very literal format.

1. The value is used as is and not modified at all. For example if the value is surrounded by quotes (as is often the case of shell variables), the quotes are included in the value passed

2. Lines beginning with # are treated as comments and are ignored

3. Blank lines are also ignored.

4. "If no = is provided and that variable is…exported in your local environment," docker "passes it to the container"
5. Thankfully, whitespace before the = will cause the run to fail

so, for example, this env-file:

# This is a comment, with an = sign, just to mess with us
VAR1=value1
VAR2=value2

USER
VAR3=is going to = trouble
VAR4=this $sign will mess with things
VAR5=var # with what looks like a comment
#VAR7 =would fail
VAR8= but what about this?
VAR9="and this?"

results in these env variables in the container:

user=ubuntu
VAR1=value1
VAR2=value2
VAR3=is going to = trouble
VAR4=this $sign will mess with things
VAR5=var # with what looks like a comment
VAR8= but what about this?
VAR9="and this?"

The bright side is that once I know what I'm working with, it's pretty easy to predict the effect. What I see is what I get. But I don't think bash would be able to interpret this in the same way without a lot of changes. How can I put this square Docker peg into a round Bash hole?

Answer 1

tl;dr:

source <(sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list)

---

You're probably going to want to source a file, whose contents

are executed as if they were printed at the command line.

But what file? The raw docker env-file is inappropriate, because it won't export the assigned variables such that they can be used by child processes, and any of the input lines with spaces, quotes, and other special characters will have undesirable results.

Since you don't want to hand edit the file, you can use a stream editor to transform the lines to something more bash-friendly. I started out trying to solve this with one or two complex Perl 5 regular expressions, or some combination of tools, but I eventually settled on one sed command with one simple and two extended regular expressions:

sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list

This does a lot.

• The first expression prepends export to any line whose first character is anything but #.
  • As discussed, this makes the variables available to anything else you run in this session, your whole point of being here.
• The second expression simply inserts a single-quote after the first = in a line, if applicable.
  • This will always enclose the whole value, whereas a greedy match could lop off some of (e.g.) VAR3, for example
• The third expression appends a second quote to any line that has at least one =.
  • it's important here to match on the = again so we don't create an unmatched quotation mark

Results:

# This is a comment, with an =' sign, just to mess with us'
export VAR1='value1'
export VAR2='value2'

export USER
export VAR3='is going to = trouble'
export VAR4='this $sign will mess with things'
export VAR5='var # with what looks like a comment'
#VAR7 ='would fail'
export VAR8=' but what about this?'
export VAR9='"and this?"'

Some more details:

• By wrapping the values in single-quotes, you've
  • prevented bash from assuming that the words after the space are a command
  • appropriately brought the # and all succeeding characters into the VAR5
  • prevented the evaluation of $sign, which, if wrapped in double-quotes, bash would have interpreted as a variable

Finally, we'll take advantage of process substitution to pass this stream as a file to source, bring all of this down to one line of bash.

source <(sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list)

Et voilà!