Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can I force pip to make a shallow checkout when installing from git?

Tags:

git

python

pip

The following command installs a Python package from a git repository:

$ pip install git+ssh://[email protected]/username/repo.git
Collecting git+ssh://[email protected]/username/repo.git
  Cloning ssh://[email protected]/username/repo.git to /tmp/pip-req-build-8s4nci15

I'm not 100% certain, but as it takes quite long I guess it clones every commit. But for installation, I only need the latest.

Instead, I would like a shallow clone (with --depth 1). Is that possible?

like image 576
Martin Thoma Avatar asked Oct 25 '18 12:10

Martin Thoma


People also ask

Can I install git through pip?

You can deploy Git locally, or use it via a hosted service, such as Github, Gitlab or Bitbucket. One of the advantages of using pip together with Git is to install the latest commits of unreleased Python packages as branches from Github.

What is git shallow fetch?

If the source repository is shallow, fetch as much as possible so that the current repository has the same history as the source repository. --update-shallow. By default when fetching from a shallow repository, git fetch refuses refs that require updating . git/shallow. This option updates .

Is it possible to use pip to install a package from a private github repository?

Read the Docs uses pip to install your Python packages. If you have private dependencies, you can install them from a private Git repository or a private repository manager.

What is shallow cloning git?

Git shallow clone lets you pull down just the latest commits, not the entire repo history. So if your project has years of history, or history from thousands of commits, you can select a particular depth to pull.


2 Answers

Adding --depth 1 to the git clone command has been discussed at length and, for the time being, rejected by pip's maintainers. It appears to be a more complex issue than one might expect, particularly since it would break setuptools_scm which is widely used.

As Klaus said in the comments you can avoid the overhead of git clone by pointing to a remote archive file instead of using a Git URL:

pip install http://my.package.repo/SomePackage-1.0.4.zip
like image 162
Chris Avatar answered Oct 10 '22 16:10

Chris


In addition to work-arounds (like ZIP) that might be available depending on hosting, a generic git work-around would be to pre-create the shallow clone, and then (since your clone is itself a git repo) point PIP to the clone.

This is not ideal (hence "work-around") in that, for an automated case, you would have to script creation of the shallow clone(s) before invoking PIP, and the data you give to PIP would list the clones' URL's instead of the canonical URL's for the respective packages.

Also, for the reasons called out in the PIP 'depth=1' debate, it might end up not working for you depending on what tools you (or your dependencies) use. It seems git describe is the crux of the issue (or at least, of one common issue); so you might be able to put a tag on the one commit you hold locally as a further work-around.

like image 26
Mark Adelsberger Avatar answered Oct 10 '22 17:10

Mark Adelsberger