Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to include git dependencies in setup.py for pip installation

I need to include Python packages available via public Github repositories along with my Python (2.7) package. My package should be installable via pip using setup.py.

So far, this could be done using dependency_links in the setup.py file:

setuptools.setup(
   name="my_package",
   version="1.0",
   install_requires=[
       "other_package==1.2"
   ],
   dependency_links=[
      "https://github.com/user/other_package/tarball/master#egg=other_package-1.2"
   ]    
)

This still works when the package gets installed with the --process-dependency-links flag, but the dependency_links functionality seems to be deprecated, since:

pip install git+https://github.com/user/my_package@master#egg=my_package-1.0 --process-dependency-links

gives me the following warning:

DEPRECATION: Dependency Links processing has been deprecated and will be removed in a future release.

Is there an alternative way to include git dependencies in the setup.py file with support for pip installation?

Edit (10/17/2016) to clarify my use case:

Let's say I find a bug in other_package. I fork the respective repo on Github, fix the bug and make a pull request. My pull request doesn't get immediately accepted (or never will be because the package is no longer actively maintained). I would like to distribute my_package together with my fork of other_package and want users to be able to pip install my_package without any further knowledge about the details of this requirement and without having to provide any additional flags upon installation. Users of my_package should further be able to include my_package as a requirement in their own custom packages.

How can this be achieved bearing compatibly with different modes of installation (wheels, eggs, develop, ...) in mind?

like image 698
mdh Avatar asked Oct 15 '16 23:10

mdh


People also ask

Does pip install dependencies of dependencies?

Pip will not flag dependency conflicts. As a result, it will happily install multiple versions of a dependency into your project, which will likely result in errors.

Can I install git through pip?

You can deploy Git locally, or use it via a hosted service, such as Github, Gitlab or Bitbucket. One of the advantages of using pip together with Git is to install the latest commits of unreleased Python packages as branches from Github.

How do I resolve dependencies in pip?

Unfortunately, pip makes no attempt to resolve dependency conflicts. For example, if you install two packages, package A may require a different version of a dependency than package B requires. Pip can install from either Source Distributions (sdist) or Wheel (. whl) files.


2 Answers

I ran into this exact issue (found a bug in someone else's project that mine depended on, made a pull request but didn't have time to wait for them to merge).

I solved it by adding this line to install_requires:

'my-package @ https://github.com/user/my-package/archive/master.tar.gz'
like image 75
Zach Avatar answered Sep 22 '22 08:09

Zach


Personally, I would avoid including git repositories as dependencies. In the scenarios you describe, I see two options.

Where Package is Unmaintained

If a package is unmaintained, you can either fork the project and distribute your own version, or you can distribute the forked code as as a submodule of your own code (i.e. include the external dependency directly in your distributable package)

Personally I prefer distributing my own version.

Where the package has yet to include your bugfix

In this case, I would distribute the fixed code as a part of your package until such a time as the bug is fixed.

like image 43
Philip Adler Avatar answered Sep 24 '22 08:09

Philip Adler