How to get PyPI to automatically install dependencies [duplicate]

Question

How can I publish a package on PyPI such that all dependencies are automatically installed, rather than manually by the user.

I specify the dependencies in setup.py with install_requires as follows:

setuptools.setup(name='myPackage',
    version='1.0',
    packages=setuptools.find_packages(),
    include_package_data=True,
    classifiers=[
        'Programming Language :: Python :: 3',
        'Operating System :: OS Independent',
        'Topic :: Scientific/Engineering :: Bio-Informatics'
                ],
    install_requires=['numpy', 'pandas', 'sklearn'],
    python_requires='>=3'
        )

And I have a requirements.txt file which is included in my MANIFEST.in:

numpy==1.15.4
sklearn==0.20.1
pandas==0.23.4

However, after publishing on test.pypi when I try to install the package, I get the following error:

Could not find a version that satisfies the requirement numpy (from myPackage==1.0.0) (from versions: )
No matching distribution found for sklearn (from myPackage==1.0.0)

This means that PyPI does not install the numpy dependency. How do I enable automatic installation of my dependencies? Should I use a virtual environment when building and publishing the package? How do I do this?

P.S. I am entirely new to this so I will appreciate explicit code or links to simple tutorial pages. Thank you.

Answer 1

I realized that installing packages from test.PyPI does not install all packages, since some of these packages are hosted on PyPI and not test.PyPI.

When I published the package on PyPI as a pre-release version (1.0a1), instead on test.PyPI, the dependencies were correctly installed. Hence, the problem was purely with test.PyPI.

Answer 2

You can specify multiple indexes via --extra-index-url. Point it to TestPyPI so your package is pulled from there, the deps from PyPI:

$ pip install myPackage --extra-index-url=https://test.pypi.org/simple/

However, the real root for the issue is that you have included the wrong dist name for the scikit-learn package. Replace sklearn with scikit-learn:

setup(
    ...,
    install_requires=['numpy', 'pandas', 'scikit-learn'],
)

Answer 3

This is an unfortunate (and known) downside to TestPyPI: The issue is that sklearn does not exist on TestPyPI, and by installing your package from there, you are telling pip to look for dependencies there as well.

Instead, you should publish to PyPI instead, and use a pre-release version so as not to pollute your versions. You can delete these pre-releases from the project later.

Answer 4

This is my approach.

I like to use a requirements.txt file instead of putting dependencies in install_requires because it's easier during dev to run:

$ pip install -r requirements.txt

To have pip install dependencies automatically, I include at the top of setup.py before setuptools.setup():

requirements = []
with open('requirements.txt', 'r') as fh:
    for line in fh:
        requirements.append(line.strip())

Then in setuptools.setup():

install_requires = requirements

To install:

pip install --index-url https://test.pypi.org/simple/ --upgrade --no-cache-dir --extra-index-url=https://pypi.org/simple/ <<package name>>

--index-url is telling pip to use the test version of pypi.

--upgrade forces an upgrade if a previous version is installed.

--no-cache-dir resolves issues with caching if doing a very quick re-release (pip doesn't pick up the new version)

--extra-index tells pip to look in the prod version of pypi if it can't find the required package in test (i.e. solves problems of dependencies not being available in test)