Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to maintain pip install options in requirements file made by pip freeze?

Tags:

python

pip

I use a pip requirements file to maintain a list of dependencies for my projects.

I now find myself having to install a library using pip install --install-option='some-setup.py.option' but pip freeze doesn't record these options in its output which I save in my project's requirements.txt. This causes problems because a simple pip install -r requirements.txt on a new machine installs all the dependencies without supplying the required arguments for this one library, and I've lost the simple round-trip operation.

So, my 2 part question is:

  • Is there a way to maintain pip install options in the pip freeze output somehow?
  • If not, is there a way to manually hack the requirements file to add the install option? I'm ok with losing the round trip nature of pip freeze if I have to, and switching to manual maintenance of the requirements file. I've checked the documentation but couldn't see anything to help.

Unnecessary but possibly interesting details follow

I want to install pymongo but without building the C extension so I can use it asynchronously in an eventlet based app.

Install as desired and build requirements.txt:

(test)day@office:~/test$ pip install pymongo --install-option='--no_ext' Downloading/unpacking pymongo   Downloading pymongo-2.1.1.tar.gz (199Kb): 199Kb downloaded   Running setup.py egg_info for package pymongo Installing collected packages: pymongo   Running setup.py install for pymongo Successfully installed pymongo Cleaning up... (test)day@office:~/test$ pip freeze >  requirements.txt (test)day@office:~/test$ cat requirements.txt bottle==0.10.7 distribute==0.6.10 eventlet==0.9.16 greenlet==0.3.3 lxml==2.3.3 pymongo==2.1.1 simplejson==2.3.2 wsgiref==0.1.2 

In new virtualenv, try to install same project from requirements.txt. pip builds the C extension for pymongo :(

(test2)day@office:~/test2$ pip install -r requirements.txt  ... Downloading/unpacking pymongo==2.1.1 (from -r requirements.txt (line 6))   Downloading pymongo-2.1.1.tar.gz (199Kb): 199Kb downloaded   Running setup.py egg_info for package pymongo Installing collected packages: pymongo   Running setup.py install for pymongo     building 'bson._cbson' extension     gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/_cbsonmodule.c -o build/temp.linux-i686-2.6/bson/_cbsonmodule.o ... Successfully installed pymongo Cleaning up... 

Update Issue 271 was opened against pip in April 2011 asking for the ability to specify per-line --install-option in requirements.txt. Please vote for the issue if you have the same problem.

like image 950
Day Avatar asked Jan 16 '12 00:01

Day


People also ask

Does pip freeze show dependencies?

Since pip freeze shows all dependencies as a flat list, finding out which are the top level packages and which packages do they depend on requires some effort. It's also tedious to resolve conflicting dependencies that could have been installed because older version of pip didn't have true dependency resolution [1].

What is the difference between pip freeze and pip list?

Therefore, you should use pip list and pip freeze as follows: If you want to check a list of packages with various conditions, use pip list . If you want to create requirements. txt , use pip freeze .


2 Answers

Since version 7.0 (released 2015-05-21), pip has the ability to parse --install-option and --global-option from requirement files. It should now be possible to have the following line in your requirements.txt:

pymongo==2.1.1 --install-option='--no_ext' 

More information can be found here and here.

like image 120
gvalkov Avatar answered Sep 24 '22 06:09

gvalkov


This might probably be too naïve, but if you're fine with managing the requirements and corresponding options manually -- why not maintain them as a small shell script that includes the entire pip incantation? Just a work-around until the feature request is listened to :)

like image 22
fgeller Avatar answered Sep 21 '22 06:09

fgeller