Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I use git repos as dependencies for my PyPi package?

Tags:

git

python

pip

pypi

I have a package which I'm pushing to PyPi and some of the depedencies are not packages, but installable git repositories. My requirements.txt looks like this

sphinx_bootstrap_theme>=0.6.5
matplotlib>=2.2.0
numpy>=1.15.0
sphinx>=1.7.5
sphinx-argparse>=0.2.2
tensorboardX
tqdm>=4.24.0
Cython>=0.28.5

# git repos
git+git://github.com/themightyoarfish/svcca-gpu.git

Accordingly, my setup.py has this content:

#!/usr/bin/env python

from distutils.core import setup
import setuptools
import os

with open('requirements.txt', mode='r') as f:
    requirements = f.read()
    required_pkgs, required_repos = requirements.split('# git repos')
    required_pkgs = required_pkgs.split()
    required_repos = required_repos.split()

with open('README.md') as f:
    readme = f.read()

setup(name=...
      ...
      packages=setuptools.find_packages('.', include=[...]),
      install_requires=required_pkgs,
      dependency_links=required_repos,
      zip_safe=False,   # don't install egg, but source
)

But running pip install <package> does not actually install the git dependency. I assume that pip doesn't actually use the setup script. It works when I run python setup.py install manually.

Edit:

I also tried removing dependency_links and just using install_requires with the repository, but when installing my repository from GitHub (the project including the above files), I'm met with

    Complete output from command python setup.py egg_info:
error in ikkuna setup command: 'install_requires' must be a string or 
list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'+git://g'"

It has been suggested in other answers that one can put something like

git+https://github.com/themightyoarfish/svcca-gpu.git#egg=svcca

into requirements.txt, but that fails with

   error in <pkg> setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'+https:/'

Question: (How) Can I list git repositories as dependencies for a pip package?

like image 721
oarfish Avatar asked Feb 26 '19 14:02

oarfish


People also ask

Can you pip install a git repo?

You can use pip to install directly from a git repository. To install flask the shortest version of the command is pip install git+https://github.com/pallets/flask.git . Notice how pip has stored the specific commit hash used when installing.

How do I create a dependency in GitHub?

On GitHub.com, navigate to the main page of the repository. Under your repository name, click Insights. In the left sidebar, click Dependency graph. Optionally, under "Dependency graph", click Dependents.


1 Answers

Out of the 50 or so different ways to specify git dependencies for Pip, the only one that did what I intended was this one (outline in PEP 508):

svcca @ git+ssh://[email protected]/themightyoarfish/svcca-gpu

This can be used in install_requires, which solves the issue of dependency_links being ignored by pip.

An amusing side-effect is that the package cannot be uploaded to PyPi with such a dependency:

HTTPError: 400 Client Error: Invalid value for requires_dist. Error: Can't have direct dependency: 'svcca @ git+ssh://[email protected]/themightyoarfish/svcca-gpu' for url: https://upload.pypi.org/legacy/
like image 135
oarfish Avatar answered Nov 03 '22 15:11

oarfish