Python can't find packages in Virtual Environment

Question

I'm trying to setup my environment for a project but python isn't able to find the modules I've installed with pip.

I did the following:

mkdir helloTwitter
cd helloTwitter
virtualenv myenv
Installing setuptools, pip, wheel...done.
source myenv/bin/activate

pip install tweepy
Collecting tweepy
  Using cached tweepy-3.5.0-py2.py3-none-any.whl
Collecting six>=1.7.3 (from tweepy)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting requests>=2.4.3 (from tweepy)
  Using cached requests-2.11.1-py2.py3-none-any.whl
Collecting requests-oauthlib>=0.4.1 (from tweepy)
  Using cached requests_oauthlib-0.6.2-py2.py3-none-any.whl
Collecting oauthlib>=0.6.2 (from requests-oauthlib>=0.4.1->tweepy)
Installing collected packages: six, requests, oauthlib, requests-oauthlib, tweepy
Successfully installed oauthlib-2.0.0 requests-2.11.1 requests-oauthlib-0.6.2 six-1.10.0 tweepy-3.5.0

When I try to import the module it says it cannot be found.

The first entry in $PATH is helloTwitter/myenv/bin All the packages are showing up in the environments site-packages directory. I seem to be using the right python and pip. Which python outputs helloTwitter/myenv/bin/python Which pip outputs helloTwitter/myenv/bin/pip

Any advice on where I'm going wrong?

Answer 1

It looks like you're manually setting your $PATH to point to your virtual environment. The whole point of the myenv/bin/activate script is to take care of this for you.

Once you have activated your virtual environment, any package you install using pip will be placed in the relevant venv site-packages directory (in your case, myenv/lib/python2.7/site-packages). Things like pip --user are unnecessary when you are working in a virtual environment (assuming default behaviour). It's all automatic.

After running activate, you can check the python binary you are using with find -iname tweepy.

Aliases can cause issues too. which is an external command, and won't always pick these up. A type -a python will flush these out.

A quick test can be done by running helloTwitter/myenv/bin/python -c 'import tweepy' directly. If this behaves differently to however you are currently running python (i.e. doesn't throw an import exception), then this is your problem.

Hope that helps.

Answer 2

Ok, I think I found a solution, if not an answer.

1. I uninstalled the package and made sure it was not in system or user.
2. I re-created the virtual environment.
3. I checked that the environments python and pip were being used.
4. This time when installing my package I added the --no-cache-dir option. The packages install successfully. Now Python can find the package.

---

derptop:environmentScience Marcus$ python 
>>> from tweepy import StreamListener 
>>> StreamListener 
<class tweepy.streaming.StreamListener'>

I checked the sys.path and it now includes the site-packages directory from the virtual environment, where previously it was absent.

Output of sys.path:

['', ....'/Users/Marcus/CodeProjects/environmentScience/myenv/lib/python2.7/site-packages']

As far as I can tell the sys.path was referencing the wrong site packages directory. Though I'm not sure why. I'm wondering if pips use of the cache was causing the site-packages reference to reset to system.