How should I generate requirements.txt? Pip Freeze not a good way

Question

How should I generate requirements.txt for Python projects?

Here is the problem I am having with pip freeze. Suppose my package P requires A, B, C. Suppose C is a library that imports X, Y, Z, but only X is needed by P. Then if I:

1) Install A
2) Install B
3) Install C, which installs X, Y, Z
4) Do a pip freeze into P's requirements.txt 

Then P's requirements.txt will look like:

1) A
2) B
3) C
4) X
5) Y
6) Z

But Y and Z are not actually required in my Python installation for P to run.

Many of the answers assume that, Y must be. However, python is a dynamic language. It is very often the case that, for example C is a huge library that uses numpy or pandas for some functionality; but P doesn't call that part of the library - in this case, I don't really need to pull those in if I know what parts of C that P needs. If all libraries were "small"; this would be rare, however there are a lot of "kitchen sink" libraries.

As far as I can tell, running pip freeze to generate P's requirements will show you all dependencies of dependencies, and thus is a superset of P's actual dependencies.

Answer 1

1. Install the pipreqs library (e.g. conda install -c conda-forge pipreqs)
2. Change dir to the project folder (cd your/repository)
3. Run the command pipreqs --force

Or just pipreqs --force your/repository.

See additional information in the official source: https://pypi.org/project/pipreqs/

Answer 2

There is a python module called pipreqs . It generates requirements.txt based on imports in the project.