Menu
We're using an internally hosted PyPI server (devpi-server) so that we can host binary wheels of huge packages that take a long time to install from source like scipy, matplotlib, etc. Installing these packages with pip install scipy works perfectly and definitely uses the wheel we've created. However, working with any of our internally developed python packages that have a dependency on one of these packages and running python setup.py install|develop|test|whatever results in the following error:
No local packages or download links found for scipy
Traceback (most recent call last):
File "setup.py", line 136, in <module>
'develop': DevelopCommand
File "/usr/local/lib/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/setuptools/dist.py", line 239, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/setuptools/dist.py", line 263, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/pkg_resources.py", line 564, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/pkg_resources.py", line 802, in best_match
return self.obtain(req, installer) # try and download/install
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/pkg_resources.py", line 814, in obtain
return installer(requirement)
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_egg
return cmd.easy_install(req)
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 587, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('scipy')
And with easy_install:
$ easy_install scipy
Searching for scipy
Reading http://pypi.internal.example.com/us/base/+simple/scipy/
No local packages or download links found for scipy
error: Could not find suitable distribution for Requirement.parse('scipy')
If I grab the URL it's looking at I get:
$ curl http://pypi.internal.example.com/us/base/+simple/scipy/
<html>
<head>
<title>us/base: links for scipy</title></head>
<body>
<h1>us/base: links for scipy</h1>
<form action="http://pypi.internal.example.com/us/base/+simple/scipy/refresh" method="post"><input name="refresh" type="submit" value="Refresh PyPI links"/></form>
us/external <a href="../../../external/+f/c48/5006bc28a8607/scipy-0.14.0-cp27-none-linux_x86_64.whl#md5=c485006bc28a8607b2fc1331df452dc1">scipy-0.14.0-cp27-none-linux_x86_64.whl</a><br/>
</body></html>
If I request the URL listed in that output, I get the wheel:
$ curl --silent 'http://pypi.internal.example.com/us/external/+f/c48/5006bc28a8607/scipy-0.14.0-cp27-none-linux_x86_64.whl#md5=c485006bc28a8607b2fc1331df452dc1' \
| file -
/dev/stdin: Zip archive data, at least v2.0 to extract
841
Download Dependencies OnlyUse the pipdeptree utility to gather a list of all dependencies, create a requirements. txt file listing all the dependencies, and then download them with the pip download command. Get the list of dependencies for a package from the setup.py file.
Python package installed with pip , e.g. WheelPython dependencies can be specified as dependencies in your packaging, and automatically installed by pip . You can include third party C libraries in wheels, but for sufficiently complex dependencies that won't work.
whl) files. Wheels are a binary installable format that includes the package and its dependencies. If both sdisk and wheel formats are available on PyPI, pip will prefer a compatible wheel. If pip does not find a wheel to install, it will automatically build a wheel and install it for you.
First of all, the issue described in the question does not depend on using devpi and can be reproduced with any PyPI server, including the main repo at pypi.org. Example:
# setup.py
from setuptools import setup
setup(name='spam', install_requires=['vprof>0.37'])
(Instead of vprof, you can take any other package that does not ship anything besides wheels)
Test it:
$ pip install --upgrade "setuptools<38.2.0"
...
Successfully installed setuptools-38.1.0
$ python setup.py install
running install
running bdist_egg
running egg_info
writing spam.egg-info/PKG-INFO
...
Processing dependencies for spam==0.0.0
Searching for vprof>0.37
Reading https://pypi.python.org/simple/vprof/
No local packages or working download links found for vprof>0.37
error: Could not find suitable distribution for
Requirement.parse('vprof>0.37')
Bang. Things get worse when you declare a binary-only package as a build dependency:
setup(name='spam', setup_requires=['vprof>0.37'])
Now all of the build and packaging commands will also fail, being unable to download build deps.
The issue solely depends on setuptools version used. Since 26 Nov 2017 and version 38.2.0, setuptools supports fetching and installing wheel dependencies, so if you still experience this issue:
setuptools
Newer OS releases should already ship a recent version of setuptools. For example, Ubuntu 18.04 has setuptools==39.0.1 by default (link). If you still have the old setuptools installed, most of the time it will be managed by the system package manager, so you shouldn't update it via pip. You can user-install an additional copy of setuptools
$ pip install --user --upgrade "setuptools>=38.2.0"
or use virtual environments for that matter.
134
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
