How to use an existing Environment variable in subprocess.Popen()

Question

Scenario

In my python script I need to run an executable file as a subprocess with x number of command line parameters which the executable is expecting.

Example:

• EG 1: myexec.sh param1 param2
• EG 2: myexec.sh param1 $MYPARAMVAL

The executable and parameters are not known as these are configured and retrieved from external source (xml config) at run time.

My code is working when the parameter is a known value (EG 1) and configured, however the expectation is that a parameter could be an environment variable and configured as such, which should be interpreted at run time.(EG 2)

In the example below I am using echo as a substitute for myexec.sh to demonstrate the scenario. This is simplified to demonstrate issue. 'cmdlst' is built from a configuration file, which could be any script with any number of parameters and values which could be a value or environment variable.

test1.py

import subprocess
import os

cmdlst = ['echo','param1','param2']

try:
    proc = subprocess.Popen(cmdlst,stdout=subprocess.PIPE)
    jobpid = proc.pid
    stdout_value, stderr_value = proc.communicate()
except (OSError, subprocess.CalledProcessError) as err:
    raise

print stdout_value

RESULT TEST 1

python test1.py

--> param1 param2

test2.py

import subprocess
import os

cmdlst = ['echo','param1','$PARAM']

try:
    proc = subprocess.Popen(cmdlst,stdout=subprocess.PIPE)
    jobpid = proc.pid
    stdout_value, stderr_value = proc.communicate()
except (OSError, subprocess.CalledProcessError) as err:
    raise

print stdout_value

RESULT TEST 2

export PARAM=param2 echo $PARAM

--> param2 python test2.py

--> param1 $PARAM

I require Test 2 to produce the same result as Test 1, considering that $PARAM would only be known at run-time and need to be retrieved from the current environment.

I welcome your advice.

Answer 1

If you want to have the shell expand environment variables, you have to set shell=True

subprocess.Popen('echo param1 $PARAM', shell=True, stdout=subprocess.PIPE)

Alternatively, you could just query the environment variable yourself when constructing the command, and then there is no need for shell expansion

subprocess.Popen(['echo', 'param1', os.environ['PARAM']], stdout=subprocess.PIPE)