Why can't `virtualenv` find `pkg_resources`?

Question

I'm trying to use virtualenv in Ubuntu to install a local virtual Python environment. When I run the shell command:

$ virtualenv ./virt_python

It throws an exception that it can't import pkg_resources. But when I open a Python shell and from pkg_resources import load_entry_point it runs fine. For reference, the complete stacktrace is below.

$ virtualenv ./virt_python
New python executable in ./virt_python/bin/python
Installing setuptools............done.
Installing pip.......
  Complete output from command /home/rpsharp/local/...hon/bin/easy_install /usr/local/lib/pytho...pport/pip-1.1.tar.gz:
  Traceback (most recent call last):
  File "/home/rpsharp/local/workspace/invest-natcap.invest-3/virt_python/bin/easy_install", line 5, in <module>
    from pkg_resources import load_entry_point
ImportError: No module named pkg_resources
----------------------------------------
...Installing pip...done.
Traceback (most recent call last):
  File "/usr/local/bin/virtualenv", line 9, in <module>
    load_entry_point('virtualenv==1.7.1.2', 'console_scripts', 'virtualenv')()
  File "/usr/local/lib/python2.6/dist-packages/virtualenv.py", line 928, in main
    never_download=options.never_download)
  File "/usr/local/lib/python2.6/dist-packages/virtualenv.py", line 1042, in create_environment
    install_pip(py_executable, search_dirs=search_dirs, never_download=never_download)
  File "/usr/local/lib/python2.6/dist-packages/virtualenv.py", line 640, in install_pip
    filter_stdout=_filter_setup)
  File "/usr/local/lib/python2.6/dist-packages/virtualenv.py", line 1006, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/rpsharp/local/...hon/bin/easy_install /usr/local/lib/pytho...pport/pip-1.1.tar.gz failed with error code 1

I tried the solution proposed here https://stackoverflow.com/a/10538412/42897 but it didn't have any effect.

Answer 1

I had the same problem when trying to run virtualenv, found out the virtualenv was installed in /home/{user}/install/lib/python2.7/site-packages while the python was pointing to /home/{user}/install/bin/virtualenv - you should know this by running

which virtualenv

So I had to uninstall and reinstall virtualenv

pip uninstall virtualenv 
pip install virtualenv

This worked for me.

Answer 2

The problem is that recent versions never download neither setuptools (distribute) nor pip and expect to find their wheels locally. Usually virtualenv says something like

Cannot find a wheel for setuptools
Cannot find a wheel for pip

and fails with the ImportError after that. This is documented:

If no satisfactory local distributions are found, virtualenv will fail. Virtualenv will never download packages.

You may want to check if you have VIRTUALENV_EXTRA_SEARCH_DIR set in your environment or the corresponding option in virtualenv's config file and disable this.

To find out where virtualenv actually searches for the packages you can temporarily add either print statements in /usr/local/lib/python2.6/dist-packages/virtualenv.py or somethig like import pdb; pdb.set_trace(). The function in question is find_wheels and you make it look something like this:

def find_wheels(projects, search_dirs):
    # … skipping docstring and comments
    for project in projects:
        for dirname in search_dirs:
            print '*** search_dir:', dirname
            files = glob.glob(os.path.join(dirname, project + '-*.whl'))
            if files:
                wheels.append(os.path.abspath(files[0]))
                break
        else:
            logger.fatal('Cannot find a wheel for %s' % (project,))

    return wheels