Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How does Python / pip handle conflicting transitive dependencies?

Suppose I want to install package a which requires the packages b1 and b2. In turn, b1 requires c > 1.0.0 and b2 requires c < 1.0.0. So the requirements of b1 and b2 cannot be fulfilled at the same time with the same package.

In principle / other programming languages, this is not a problem. One could install two versions of c side by side and make sure that b1 uses another version than b2.

However, I'm not sure if pip can install two versions of the same package. My first question is: Can pip install two versions of one package?

My main question is how one actually can deal with that problem. The only ways I can imagine right now is to

  1. fork b1 (or b2) and a version of c that works for the fork, and upload b1_forked and c_for_b1_forked to PyPI, or
  2. Include the code of b1 (or b2) directly in my project

Both seem more problematic than necessary.

What I tried

>>> import natsort; print(natsort.__file___)
'/home/moose/.local/lib/python3.6/site-packages/natsort/__init__.py'

$ cd /home/moose/.local/lib/python3.6/site-packages
$ ls
[... a lot of *.dist-info directories, some .py files, some .so files, ]
[... some directories called like the packages I've installed]

So I'm pretty sure this is where Python looks for installed packages and that only one version is installed (although the *-dist-info directories confuse me a bit).

This blog post suggests that there is no good solution for conflicting transitive dependencies at the moment. Do other projects (e.g. poetry) help with that?

like image 320
Martin Thoma Avatar asked Feb 05 '20 21:02

Martin Thoma


People also ask

Does pip dependency resolution?

Pip does not provide true dependency resolution, but this can be solved by using it in conjunction with a requirements. txt file. Requirements. txt files can be used to make pip resolve dependency conflicts between different packages.

How do I fix the conflict in Python?

You may see a version conflict if your dependencies are demanding a specific version of a library that you don't currently use in your own requirements. txt. The best solution for this is to pin your requirements. txt file to a version of the library that works for both your project and for your dependencies.

How do I use pip conflict checker?

Usage. Simply run the command pipconflictchecker. If any dependency conflicts are found an output dump of all conflicts will be shown, and an exit code of 1 will be returned.


1 Answers

In principle / other programming languages, this is not a problem. One could install two versions of c side by side and make sure that b1 uses another version than b2.

That's not a solution. If c manages a shared resource (console, e.g.) sooner or later b1 and b2 will stomp each other input or output via different cs and you end up with incorrect input and garbage output.

What you describe is a general problem, not limited to Python or pip. The only solution is to change b1 and/or b2 to agree on a version of c. Either downgrade b1 to allow c < 1.0 or upgrade b2 to allow c > 1.0.

Can pip install two versions of one package?

No, and the problem is not in pip but in Python: its import system doesn't allow importing from different versions of the same package. You can look at mitsuhiko/multiversion (Python2-only).

like image 157
phd Avatar answered Sep 17 '22 21:09

phd