Set Environmental Variables in Python with Popen

Question

I want to set an environmental variable in linux terminal through a python script. I seem to be able to set environmental variables when using os.environ['BLASTDB'] = '/path/to/directory' .

However I was initially trying to set this variable with subprocess.Popen with no success.

import subprocess
import shlex

cmd1 = 'export BLASTDB=/path/to/directory'
args = shlex.split(cmd1)
p = subprocess.Popen(args, stdout=subprocess.PIPE).communicate()

Why does subprocess.Popen fail to set the environmental variable BLASTDB to '/path/to/directory'?

NOTE: This also fails when using:

import os
os.system('export BLASTDB=/path/to/directory')

Answer 1

Use the env parameter to set environment variables for a subprocess:

proc = subprocess.Popen(args, stdout=subprocess.PIPE,
                        env={'BLASTDB': '/path/to/directory'})

Per the docs:

If env is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of inheriting the current process’ environment, which is the default behavior.

Note: If specified, env must provide any variables required for the program to execute. On Windows, in order to run a side-by-side assembly the specified env must include a valid SystemRoot.

---

os.environ can used for accessing current environment variables of the python process. If your system also supports putenv, then os.environ can also be used for setting environment variables (and thus could be used instead of Popen's env parameter shown above). However, for some OSes such as FreeBSD and MacOS, setting os.environ may cause memory leaks, so setting os.environ is not a robust solution.

---

os.system('export BLASTDB=/path/to/directory') runs a subprocess which sets the BLASTDB environment variable only for that subprocess. Since that subprocess ends, it has no effect on subsequent subprocess.Popen calls.

Answer 2

As far as I know, you cannot really modify the executing process' environment from a subprocess or subshell, be it Python or bash itself. Environment variables are specific to the particular process you are on (at least on Unix, which you seem to be using).

Any child process spawned will usually inherit that environment, but only a copy of it. For instance, if you run bash from inside your terminal session and export a new environment variable, once you exit that subshell, your original shell will be untouched. Running Python is no different.