Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use an existing Environment variable in subprocess.Popen()

Scenario

In my python script I need to run an executable file as a subprocess with x number of command line parameters which the executable is expecting.

Example:

  • EG 1: myexec.sh param1 param2
  • EG 2: myexec.sh param1 $MYPARAMVAL

The executable and parameters are not known as these are configured and retrieved from external source (xml config) at run time.

My code is working when the parameter is a known value (EG 1) and configured, however the expectation is that a parameter could be an environment variable and configured as such, which should be interpreted at run time.(EG 2)

In the example below I am using echo as a substitute for myexec.sh to demonstrate the scenario. This is simplified to demonstrate issue. 'cmdlst' is built from a configuration file, which could be any script with any number of parameters and values which could be a value or environment variable.

test1.py

import subprocess
import os

cmdlst = ['echo','param1','param2']

try:
    proc = subprocess.Popen(cmdlst,stdout=subprocess.PIPE)
    jobpid = proc.pid
    stdout_value, stderr_value = proc.communicate()
except (OSError, subprocess.CalledProcessError) as err:
    raise

print stdout_value

RESULT TEST 1

python test1.py

--> param1 param2

test2.py

import subprocess
import os

cmdlst = ['echo','param1','$PARAM']

try:
    proc = subprocess.Popen(cmdlst,stdout=subprocess.PIPE)
    jobpid = proc.pid
    stdout_value, stderr_value = proc.communicate()
except (OSError, subprocess.CalledProcessError) as err:
    raise

print stdout_value

RESULT TEST 2

export PARAM=param2 echo $PARAM

--> param2 python test2.py

--> param1 $PARAM

I require Test 2 to produce the same result as Test 1, considering that $PARAM would only be known at run-time and need to be retrieved from the current environment.

I welcome your advice.

like image 817
user3927078 Avatar asked May 27 '16 16:05

user3927078


People also ask

Does subprocess inherit environment variables?

4 Environment Variables What makes the environment useful is that subprocesses inherit the environment automatically from their parent process. This means you can set up an environment variable in your login shell, and all the programs you run (including Emacs) will automatically see it.

How do I use subprocess Popen in Python?

The recommended approach to invoking subprocesses is to use the run() function for all use cases it can handle. For more advanced use cases, the underlying Popen interface can be used directly. Run the command described by args. Wait for command to complete, then return a CompletedProcess instance.

Should I use Popen or subprocess?

The main difference is that subprocess. run() executes a command and waits for it to finish, while with subprocess. Popen you can continue doing your stuff while the process finishes and then just repeatedly call Popen. communicate() yourself to pass and receive data to your process.


1 Answers

If you want to have the shell expand environment variables, you have to set shell=True

subprocess.Popen('echo param1 $PARAM', shell=True, stdout=subprocess.PIPE)

Alternatively, you could just query the environment variable yourself when constructing the command, and then there is no need for shell expansion

subprocess.Popen(['echo', 'param1', os.environ['PARAM']], stdout=subprocess.PIPE)
like image 70
Brendan Abel Avatar answered Nov 14 '22 21:11

Brendan Abel