What does pip-compile do? What is it's use?

Question

I am a beginner in programming and Python. I read pip-compiles definition in pip-tools documentation but I could not understand. Can someone explain me this?

More specifically, what does compiling requirements.in to produce requirements.txt mean?

Answer 1

You want to be able to lock down the versions of all of the packages that your Python code depends on in your requirements.txt file. You want this file to include versions for not just the direct dependencies that your code imports directly, but also versions for all of the transitive dependencies as well...that is, the versions of modules that your directly dependent modules themselves depend on.

So the question is...how do you maintain the contents of "requirements.txt"? You can use pip freeze > requirements.txt, but this is messy. It depends not on a clear list of what the direct dependencies of your app are, but rather on what happens to be in your environment at the time of creation. What you really want is to have a file in which you list the direct dependencies of your app, along with versions for each of them, and then somehow produce the appropriate requirements.txt file from that list such that it contains exactly versions for those direct dependencies as well as versions for just the transitive dependencies needed by those direct dependencies.

The requirements.in file and pip-compile together give you this desired behavior. In requirements.in, you list just the direct dependencies of your app. Then you run pip-compile on that file to produce requirements.txt. The compile process will produce what you want...a file that contains both the modules listed in requirements.in and the transitive dependencies of those modules.

UPDATE: Someone asked why you should go through this exercise to lock down all of the versions of the packages upon which your application relies. The reason for this is that if you don't do this, then whenever you rebuild your application, you will get a built that uses the latest (ie: different) versions of some or all of the packages that it uses. So what if a change is made to one of those packages that causes your app's behavior to change? Maybe it causes an exception to be thrown, killing your app. You want to prevent this. Going through the process discussed by this question/answer locks down all of the versions of the packages that your application uses, preventing changes made to later versions of those packages from affecting the behavior of your application.