Caching remote repository for pip installs

Question

In my pi requirements file, I require specific commits of various repos, ie:

git+http://github.com/frankban/django-endless-pagination.git@725bde91db#egg=django-endless-pagination

The problem I'm having with this is that it apparently requires pip to clone the repo anew for every install, ignoring the default download cache entirely.

Are there any ways to require this repo to be cached locally? Or, alternately, what is the best solution to package this up and keep the package locally available?

Answer 1

You can do two things: use an editable install, or cache the result of the install as a wheel.

Using the -e switch causes pip to clone the repository into the src subdirectory of your virtualenv; you can then reuse that copy each time you want to re-install:

pip install -e -r requirements.txt

Pip then just re-uses the existing source each time you re-run the command (updating from git rather than pulling in a completely new copy of the repo), or, since the installation uses the actual working directory, you can just use git pull in src/django-endless-pagination instead.

You can cache the result of the pip install as a Python Wheel:

pip wheel --wheel-dir=/tmp/wheelhouse -r requirements.txt

This installs all the requirements and creates wheels for each in /tmp/wheelhouse. You can then re-use the wheelhouse for subsequent installs:

pip install --use-wheel --no-index --find-links=/tmp/wheelhouse -r requirements.txt

The wheels won't be updated from the repository however.