Anaconda does not use package from activated environment

Question

I have a conda environment, in a bash terminal, with an Intel Python Distribution interpreter. However, when importing packages, they are imported from what looks to be the user directory of the system default Python, not the environment. Take a look at the version discrepancy and the __spec__ origin of the pandas package.

 ~  $  conda activate idp
 ~  $  which python
~/anaconda3/envs/idp/bin/python
 ~  $  python
Python 3.6.8 |Intel Corporation| (default, Mar  1 2019, 00:10:45) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import pandas
>>> pandas.__version__
'0.22.0'
>>> pandas.__spec__
ModuleSpec(name='pandas', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f509e184ba8>, origin='/home/torstein/.local/lib/python3.6/site-packages/pandas/__init__.py', submodule_search_locations=['/home/torstein/.local/lib/python3.6/site-packages/pandas'])
>>> 
 ~  $  conda list | head -n 3
# packages in environment at /home/torstein/anaconda3/envs/idp:
#
# Name                    Version                   Build  Channel
 ~  $  conda list | grep pandas
pandas                    0.24.1                   py36_3    intel
 ~  $  conda env list
# conda environments:
#
base                     /home/torstein/anaconda3
idp                   *  /home/torstein/anaconda3/envs/idp
py36                     /home/torstein/anaconda3/envs/py36

When using the base environment, this does NOT happen; packages (e.g. pandas) are imported from the correct path:

 ~  $  conda deactivate
 ~  $  conda env list
# conda environments:
#
base                  *  /home/torstein/anaconda3
idp                      /home/torstein/anaconda3/envs/idp
py36                     /home/torstein/anaconda3/envs/py36

 ~  $  which python
~/anaconda3/bin/python
 ~  $  python
Python 3.7.0 (default, Oct  9 2018, 10:31:47) 
[GCC 7.3.0] :: Anaconda custom (64-bit) on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
pan>>> pandas.__version__
'0.23.4'
>>> pandas.__spec__
ModuleSpec(name='pandas', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7fad808a8e80>, origin='/home/torstein/anaconda3/lib/python3.7/site-packages/pandas/__init__.py', submodule_search_locations=['/home/torstein/anaconda3/lib/python3.7/site-packages/pandas'])

The relevant part of .bashrc (no anaconda explicitly set in path):

export PATH="/home/torstein/.cargo/bin:$PATH"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/lib/intel64_lin:/opt/intel/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin/"

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/torstein/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/torstein/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/home/torstein/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/home/torstein/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup

Which yields these $PATHs, for the base and idp envs respectively:

 ~  $  echo $PATH
/home/torstein/anaconda3/bin:/home/torstein/anaconda3/condabin:/home/torstein/.cargo/bin:/home/torstein/.cargo/bin:/home/torstein/anaconda3/bin:/home/torstein/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/torstein/.local/bin:/home/torstein/bin
 ~  $  conda activate idp
 ~  $  echo $PATH
/home/torstein/anaconda3/envs/idp/bin:/home/torstein/anaconda3/condabin:/home/torstein/.cargo/bin:/home/torstein/.cargo/bin:/home/torstein/anaconda3/bin:/home/torstein/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/torstein/.local/bin:/home/torstein/bin

The pandas that I do want to import is located here, where it should be:

/home/torstein/anaconda3/envs/idp/lib/python3.6/site-packages/pandas

Answer 1

Diagnosis

There appears to be (or have been) another Python 3.6 in the PATH, and I suspect that somehow the Conda dependency resolver ended up resolving some packages to this alternative site-packages and inadvertently including this directory in sys.path. This appears to be a known issue.

Evidence

The reason I believe this is because the pandas module is being loaded from here:

/home/torstein/.local/lib/python3.6/site-packages/pandas

If you check in Python

import sys

sys.path

I expect that this should show the above directory.

Since it was reported that PYTHONPATH is empty (as it should be!), that can't be responsible for this misloading, hence I think it was Conda that somehow configured the env this way.

Also, the fact that your Python 3.7 env is unaffected is likely because you can't load modules across different minor versions.

Immediate Solution

Somehow you need to get rid of that dependency. There are a few things to try

1. Remove that /home/torstein/.local/ from your PATH. This could cause other issues though. Presumably you have it in PATH because you have other non-development things in there.
2. Dump specifically that site-packages directory. In comments, it was stated that this is residual from a global Python installation no longer in use, so it seems like a good thing to just get rid of. Do back it up, though, in case there are other dependencies on it.
3. Clear this path from sys.path before importing modules. Not sure of a clean way to do this.

Personally, I'd want to delete it and create new envs. It can be hard to know how tied into this directory you are, so I'd be wary of assuming that other packages don't somehow have hidden dependencies on what is in there.

Long-Term Workaround

The recommended workaround from the GitHub issue is to add the following environment variable,

export PYTHONNOUSERSITE=True

which prevents Conda from loading other local site-packages directories. With this, you shouldn't have encountered the problem in the first place.