setuptools finds wrong package during install

Question

In myria-python, we use setuptools with install_requires to configure which packages are needed. In our particular setup file, we include requests-toolbelt and requests in that list.

When we create a new virtual environment and then run python setup.py install, it fails the first time with Cannot find required distribution requests. This happens seemingly because pip identifies requests toolbelt-0.3.1 (note the space) as the right match for package requests.

Running python setup.py install again seems to install requests after all.

Here is a GitHub issue with a full log of the install process.

Steps to reproduce:

• git clone https://github.com/uwescience/myria-python.git
• cd myria-python
• mkvirtualenv myria-python
• python setup.py

The entire lab seems to have this issue, however all of us use Mac OS X with either 10.9 or 10.10 installed. Here are my machine's specs:

• OS X 10.10.1
• Python 2.7.9 (default, Dec 10 2014, 23:46:04)
• pip 1.5.6
• mkvirtualenv 1.11.6

I was also able to duplicate it on one of our Ubuntu servers:

• Ubuntu 14.04.1 LTS \n \l
• Python 2.7.6
• pip 1.5.4
• mkvirtualenv 1.11.4

Here is the tail of the error log:

Searching for requests-toolbelt
Reading https://pypi.python.org/simple/requests_toolbelt/
Best match: requests-toolbelt 0.3.1
Downloading https://pypi.python.org/packages/source/r/requests-toolbelt/requests-toolbelt-0.3.1.tar.gz#md5=e563377e46cd0be8c7b3ac144a65844c
Processing requests-toolbelt-0.3.1.tar.gz
Writing /var/folders/m_/qltd_g_13qd1v5tvr4l6q2rc0000gn/T/easy_install-2lqn7g/requests-toolbelt-0.3.1/setup.cfg
Running requests-toolbelt-0.3.1/setup.py -q bdist_egg --dist-dir /var/folders/m_/qltd_g_13qd1v5tvr4l6q2rc0000gn/T/easy_install-2lqn7g/requests-toolbelt-0.3.1/egg-dist-tmp-riz25e
no previously-included directories found matching '*.pyc'
warning: manifest_maker: MANIFEST.in, line 6: 'recursive-include' expects <dir> <pattern1> <pattern2> ...

warning: manifest_maker: MANIFEST.in, line 7: 'recursive-include' expects <dir> <pattern1> <pattern2> ...

no previously-included directories found matching 'docs/_build'
zip_safe flag not set; analyzing archive contents...
Adding requests-toolbelt 0.3.1 to easy-install.pth file

Installed /Users/dhalperi/Envs/myria-python2/lib/python2.7/site-packages/requests_toolbelt-0.3.1-py2.7.egg
Searching for requests
Best match: requests toolbelt-0.3.1
Downloading https://pypi.python.org/packages/source/r/requests-toolbelt/requests-toolbelt-0.3.1.tar.gz#md5=e563377e46cd0be8c7b3ac144a65844c
Processing requests-toolbelt-0.3.1.tar.gz
Writing /var/folders/m_/qltd_g_13qd1v5tvr4l6q2rc0000gn/T/easy_install-LKxX9E/requests-toolbelt-0.3.1/setup.cfg
Running requests-toolbelt-0.3.1/setup.py -q bdist_egg --dist-dir /var/folders/m_/qltd_g_13qd1v5tvr4l6q2rc0000gn/T/easy_install-LKxX9E/requests-toolbelt-0.3.1/egg-dist-tmp-3tgz5e
no previously-included directories found matching '*.pyc'
warning: manifest_maker: MANIFEST.in, line 6: 'recursive-include' expects <dir> <pattern1> <pattern2> ...

warning: manifest_maker: MANIFEST.in, line 7: 'recursive-include' expects <dir> <pattern1> <pattern2> ...

no previously-included directories found matching 'docs/_build'
zip_safe flag not set; analyzing archive contents...
requests-toolbelt 0.3.1 is already the active version in easy-install.pth

Installed /Users/dhalperi/Envs/myria-python2/lib/python2.7/site-packages/requests_toolbelt-0.3.1-py2.7.egg
error: Could not find required distribution requests

How can I fix this so that the package installs without running setup.py twice?

Answer 1

(Mirrored from https://github.com/uwescience/myria-python/pull/35)

Multiple hours of 100's of Google searches eventually took me to this numpy thread:

https://github.com/numpy/numpy/issues/2434

for which they had to add numpy to setup_requires AND install_requires to fix a similar issue. Doing this with requests seems to have worked. However, I am somewhat skeptical as @Timusan indicated above not seeing our problem in the first place.

I tried on my Mac as well as on Ubuntu, and am now able to install from a clean virtualenv on both systems.