How to expand environment variables in python as bash does?

Question

With os.path.expandvars I can expand environment variables in a string, but with the caveat: "Malformed variable names and references to non-existing variables are left unchanged" (emphasis mine). And besides, os.path.expandvars expands escaped \$ too.

I would like to expand the variables in a bash-like fashion, at least in these two points. Compare:

import os.environ
import os.path
os.environ['MyVar'] = 'my_var'
if 'unknown' in os.environ:
  del os.environ['unknown']
print(os.path.expandvars("$MyVar$unknown\$MyVar"))

which gives my_var$unknown\my_var with:

unset unknown
MyVar=my_var
echo $MyVar$unknown\$MyVar

which gives my_var$MyVar, and this is what I want.

Answer 1

The following implementation maintain full compatibility with os.path.expandvars, yet allows a greater flexibility through optional parameters:

import os
import re

def expandvars(path, default=None, skip_escaped=False):
    """Expand environment variables of form $var and ${var}.
       If parameter 'skip_escaped' is True, all escaped variable references
       (i.e. preceded by backslashes) are skipped.
       Unknown variables are set to 'default'. If 'default' is None,
       they are left unchanged.
    """
    def replace_var(m):
        return os.environ.get(m.group(2) or m.group(1), m.group(0) if default is None else default)
    reVar = (r'(?<!\\)' if skip_escaped else '') + r'\$(\w+|\{([^}]*)\})'
    return re.sub(reVar, replace_var, path)

Below are some invocation examples:

>>> expandvars("$SHELL$unknown\$SHELL")
'/bin/bash$unknown\\/bin/bash'

>>> expandvars("$SHELL$unknown\$SHELL", '')
'/bin/bash\\/bin/bash'

>>> expandvars("$SHELL$unknown\$SHELL", '', True)
'/bin/bash\\$SHELL'

Answer 2

Try this:

re.sub('\$[A-Za-z_][A-Za-z0-9_]*', '', os.path.expandvars(path))

The regular expression should match any valid variable name, as per this answer, and every match will be substituted with the empty string.

Edit: if you don't want to replace escaped vars (i.e. \$VAR), use a negative lookbehind assertion in the regex:

re.sub(r'(?<!\\)\$[A-Za-z_][A-Za-z0-9_]*', '', os.path.expandvars(path))

(which says the match should not be preceded by \).

Edit 2: let's make this a function:

def expandvars2(path):
    return re.sub(r'(?<!\\)\$[A-Za-z_][A-Za-z0-9_]*', '', os.path.expandvars(path))

check the result:

>>> print(expandvars2('$TERM$FOO\$BAR'))
xterm-256color\$BAR

the variable $TERM gets expanded to its value, the nonexisting variable $FOO is expanded to the empty string, and \$BAR is not touched.