PIP randomly fails 'Could not find a version that satisfies the requirement' with the same requirements.txt

Question

As part of our CI testing we install a virtualenv with some pip packages from a constant requirements.txt file.

this installation process randomly fails from time to time with no apparent reason as the requirements.txt file doesn't change. And each time it's for a different random package.

The CI is on an AWS machine so I don't think it can be an internet issue

The failure looks similar to that (with different package failing):

Collecting django-rest-auth==0.9.3 (from -r requirements.txt (line 7))
Could not find a version that satisfies the requirement django-rest-auth==0.9.3 (from -r requirements.txt (line 7)) (from versions: )
No matching distribution found for django-rest-auth==0.9.3 (from -r requirements.txt (line 7))

Or

Collecting py>=1.5.0 (from pytest->-r requirements.txt (line 15))
Could not find a version that satisfies the requirement py>=1.5.0 (from pytest->-r requirements.txt (line 15)) (from versions: )
No matching distribution found for py>=1.5.0 (from pytest->-r requirements.txt (line 15))

EDIT: Tried adding --timeout 30 --retries 15 which didn't seem to change anything

Answer 1

The (from versions: ) is a big clue. It comes from this line in the pip source code. The fact that nothing comes after versions: implies that pip couldn't find any version of djanto-rest-auth in the pypi index. Never mind the version that your requirements.txt asks for. That check comes later.

Here is what versions: should look like when you try to install something that can't be found:

> pip install django-rest-auth==29.42  # ridiculous version that won't be found
Error: Could not find a version that satisfies the requirement django-rest-auth==29.42
(from versions: 0.9.4.macosx-10.14-intel, 0.1, 0.2, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 
0.2.5, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0,
0.8.1, 0.8.2, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.9.5)

The only way versions: can be blank is if the routine find_all_candidates returns an empty list. find-all-candidates ought to return a list with every version of the package that has been released on pypi.org.

I don't know why that routine randomly returns an empty list on your CI box, but I have a hunch it involves the code that scrapes https://pypi.org/simple/django-rest-auth/ for links. Perhaps it is getting back an empty HTML page every once in a while.

Here are some things you can do to debug

• Run pip with the --verbose flag. This activates some logging of URLs and links that might be helpful.
• ssh into your CI server and find site-packages/pip/_internal/index.py. Edit it and add more logging to help you debug. I would start by logging the HTML content that it gets back from pypi.org.
• if you get lucky and find a way to repeat the problem, use pdb to debug pip on your CI server.

Here is an example of the extra logging you get with --verbose. Really curious what this looks like when your glitch happens:

Collecting django-rest-auth==29.42
-------------------------------------------------------
  1 location(s) to search for versions of django-rest-auth:
  * https://pypi.org/simple/django-rest-auth/
  Getting page https://pypi.org/simple/django-rest-auth/
  Looking up "https://pypi.org/simple/django-rest-auth/" in the cache
  Request header has "max_age" as 0, cache bypassed
  Starting new HTTPS connection (1): pypi.org:443
  https://pypi.org:443 "GET /simple/django-rest-auth/ HTTP/1.1" 200 2467
  Updating cache with response from "https://pypi.org/simple/django-rest-auth/"
  Caching due to etag
  Analyzing links from page https://pypi.org/simple/django-rest-auth/
    Found link https://files.pythonhosted.org/packages/c8/ff/cffe8cb7961a1665f20115adb035d23a6b1cb08a2a6c1d6de802b13cdcc9/django-rest-auth-0.1.tar.gz#sha256=fcb9feced7f066c92a5f29f2930609316095a7abe3806e09c3d63c36c3607780 (from https://pypi.org/simple/django-rest-auth/), version: 0.1
    Found link https://files.pythonhosted.org/packages/af/d2/5d37d3f1c7055284b969e2de8eaf7d7dc16b51fba94f3325d92d053e12a8/django-rest-auth-0.2.tar.gz#sha256=04ae1a5d991692293ec95a10b517bdb26b41823a645400dc0b899d9f538013b9 (from https://pypi.org/simple/django-rest-auth/), version: 0.2
    Found link https://files.pythonhosted.org/packages/46/87/816fcc68a4552916cb82eef40dfd1bd752f831a329e927b96b7f9c6c0db7/django-rest-auth-0.2.1.tar.gz#sha256=3306e739bb8f34d47285c9e1616f75a9d8b4f6985102d68509d5aec5af62c760 (from https://pypi.org/simple/django-rest-auth/), version: 0.2.1
... about 20 more of these

Good luck. Sorry I couldn't give you an exact solution.

Answer 2

I have that problem when I have a heavy dependency, so I updated the timeout for pip and problem solved. i.e my .pip/pip.conf has a timeout of 30 seconds

[global]
timeout = 30

Answer 3

Problem: Their may be problem with your python and other libraries version. May be your django wheel require some-other library which is installed in your anaconda environment but not satisfying the versions. when you use pip command it just try to download the wheel not care about version and not if version are not matching it just give us error.

Try using conda command because conda command will update your version according to the requirement. when you you conda command it will download library for all of the environments you are using in anaconda navigator. But Pip will only install library from which environment pip command is called.

Solution: try to install this library using conda command like

conda install django-rest-auth==0.9.3

This command will help you to solve version error.