conda environment has access to system modules, how to prevent?

Question

I noticed that when I create a new enviornment with conda, I can import python modules in that environment that were NOT installed there.

Example with keras: Although the module is NOT in that enviornment:

(py2) user@user-Precision-7920-Tower:~$ conda list keras
# packages in environment at /home/user/anaconda3/envs/py2:
#
# Name                    Version                   Build  Channel

I can still import it, apparently from the system (user) install, outside conda!

(py2) user@user-Precision-7920-Tower:~$ python
Python 2.7.15 | packaged by conda-forge | (default, Mar  5 2020, 14:56:06) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import keras
Using TensorFlow backend.
>>> keras.__file__
'/home/user/.local/lib/python2.7/site-packages/keras/__init__.pyc'

In fact, python inside conda has access to non-conda paths!

>>> import sys
>>> 
>>> sys.stdout.write("\n".join(sys.path))

/home/user/anaconda3/envs/py2/lib/python27.zip
/home/user/anaconda3/envs/py2/lib/python2.7
/home/user/anaconda3/envs/py2/lib/python2.7/plat-linux2
/home/user/anaconda3/envs/py2/lib/python2.7/lib-tk
/home/user/anaconda3/envs/py2/lib/python2.7/lib-old
/home/user/anaconda3/envs/py2/lib/python2.7/lib-dynload
/home/user/.local/lib/python2.7/site-packages               <-- 
/home/user/anaconda3/envs/py2/lib/python2.7/site-packages>>> 

Conda is supposed to keep things isolated. How did this path endd up in here, and how to avoid this from happening?

UPDATE:

My user-level python is 2.7, and I noticed this behavior always happen when I create a new conda environment with python 2.7, this just automatically adds the .local/lib/python2.7/site-packages to PYTHONPATH.

If I create new conda environments with python3.x , this does not happen.

Does this mean that one cannot create a separate isolated conda environment for the same python version as the user-level python?

Answer 1

In addition to what @VikashB mentioned, these can result from packages installed with pip install --user. As @TimRoberts alluded to in the comments, the site module, which populates the sys.path variable, searches paths like ~/.local/lib/python*/site-packages by default.

Temporary Options

One can disable the site module from loading such packages (see PEP 370), either by launching Python with an -s flag (python -s) or by setting the environment variable PYTHONNOUSERSITE:

export PYTHONNOUSERSITE=1
python

Longer-term options

Hiding from site module

If you need to keep these packages for some reason, one option is to move them to a non-default location so the site module doesn't find them. For example,

mkdir ~/.local/lib/py_backup
mv ~/.local/lib/python* ~/.local/lib/py_backup

This will effectively hide them, and they could still be used through PYTHONPATH if necessary.

Removal

If you don't need the packages, and only use Conda then consider just removing them

rm -r ~/.local/lib/python*

For reference, Conda users are discouraged from using the --user flag in the Conda documentation. Conda environments assume full isolation of environments, so leakage such as OP reports can lead to undefined behavior.

Experimental: envvar-pythonnousersite-true

In response to another question, I put together a simple Conda package that sets the PYTHONNOUSERSITE=1 variable at environment activation time. There are other ways to set environment variables, but this is a quick and minimal patch.

It can be installed with:

conda install merv::envvar-pythonnousersite-true

Answer 2

There are multiple possible reasons why this might happen.

Check if you have any activation scripts that manually add these paths.

Another option is to check if you have environment variables such as PYTHONPATH or PYTHONHOME set. If they are, then check where they are being set and remove them.

You can use conda info -a in an activated environment to show all the relevant variables/information.