pip: selecting index url based on package name?

Question

I have created a local private packages repository. By convention, all those packages are named with an identifying prefix, for example foo-package. These packages may depend on public packages available on PyPi. Let's assume there's no risk of having a package in PyPi with the same name. By using --index-url together with --extra-index-url, I can make pip search on both. This will happen every single time.

Even when pip finds a package on PyPi, it will still try to find it also on the extra url. What I'd like to achieve is that pip only searches the extra url when the package name is foo-*, and only searches PyPi for everything else. Is this possible somehow?

Answer 1

As far as I understood, the philosophy from the point of view of pip, and PyPI (and I guess PyPA ecosystem in general) is that indexes should be indistinguishable, interchangeable. If 2 projects of the same name exist on 2 indexes, it should be assumed that they are the exact same project. And 2 distributions of the same name and version number should be assumed to be the exact same distribution and so it does not matter from which one we fetch. In other words:

Packages are expected to be unique up to name and version, so two wheels with the same package name and version are treated as indistinguishable by pip. This is a deliberate feature of the package metadata, and not likely to change.

-- https://github.com/pypa/pip/issues/5045#issuecomment-369521345

[Short of relying on direct URLs Library @ https://dists.tango.dev/library-1.2.3-xyz.whl I do not see how it can be done, right now. But maybe I am missing something obvious.]

If one needs to circumvent this behaviour and regain control over the situation, they need to put something like devpi or pydist in place.

• In the case of devpi, its "inheritance" feature seems of particular importance here. As far as I understood this is the key feature that would prevent downloading a dependency from the "wrong" index (not sure how exactly that works and how to do the configuration, though).
• For pydist: https://pydist.com/blog/extra-index-url
• Probably also possible in other servers...

References:

• Dependency notation including the index URL
• https://github.com/pypa/pip/issues/5045#issuecomment-369521345
• PyDist – Blog – The Problem with --extra-index-url