What are the benefits of pip and virtualenv?

Question

So everyone is telling me to use pip and virtualenv but no-one is able to explain me how it is better than my current approach. The main reason for people to use pip and virtualenv seems to be that everyone else is using it...

I'm sure there are very good reasons to use PIP and virtualenv but I haven't been able to find them with Google. I'm hoping that someone from the stackoverflow community will be able to explain them to me.

Here is how I currently organize my Django projects:

site/src/ : contains all python-only dependencies of my project  site/lib/ : contains symlinks to the python packages  site/[projectname]/ : contains all my project specific code 

The entire site folder is check in my repository (yes, including all python-only dependencies such as django itself).

All non-python-only dependencies (PIL, psycopg2, ...) are documented in a README and installed at the system level (apt-get install ....)

For example, let's say I have a project name "projectfoo" that depends on django-1.2.3, pygeoip-0.1.3 and psycopg2 I will have:

/usr/lib/python2.5/site-packages/psycopg2  ~/projects/foo/site : checkout of my repository ~/projects/foo/site/src/django-1.2.3 ~/projects/foo/site/src/pygeoip-0.1.3 ~/projects/foo/site/lib/django -> symlink to ../src/django-1.2.3/django ~/projects/foo/site/lib/pygeoip -> symlink to ../src/pygeoip-0.1.3/pygeoip ~/projects/foo/site/projectfoo/ 

Now how does this compare with PIP/virtualenv in practice?

Deploying the project with my current approach:

svn checkout https://myserver.com/svn/projectfoo/tags/1.0.0STABLE/site 

Deploying with PIP/virtualenv:

wget https://myserver.com/svn/projectfoo/tags/1.0.0STABLE/projectfoo-requirements.txt pip install -U -E projectfoo-venv -r projectfoo-requirements.txt 

Working on a project with my current approach:

cd ~/projects/foo/site/projectfoo export PYTHONPATH=.:..:../lib ./manage.py runserver 0:8000 

Working on a project with PIP/virtualenv:

workon projectfoo cd path/to/project ./manage.py runserver 0:8000 

Dealing with non-python-only dependencies:

non-python-only dependencies would be handled the same way, there is no way I'm going to use the --no-site-packages option of virtualenv and install a compiler and all the build dependencies on my servers, I don't think anyone is actually doing it anyway.

Upgrading a python-only dependency with my current approach:

I checkout/unzip the new version in src, remove the old one from src, update the symlink in lib and commit. Now everyone else working on the project will get the update at their next svn up or git pull. One thing that is not nice is that the old folder in src will remains if it contains some pyc file in it, this can easily be avoided by removing all pyc before updating your local copy.

Upgrading a python-only dependency with PIP/virtualenv:

You commit a new version of the requirements file, hopefully everyone working on the project notice the new version when they update their local copy and they then run pip install -E projectfoo-venv -r requirements.txt.

Edit: I remove the use of -U with pip, this is not needed with pip 8.2

Edit: The only advantage on pip/virtualenv over my current approach seems to be when you work on different projects requiring differents version of python. But in my experience when you need different versions of python you also need different versions of others system libraries (PIL, psycopg2, ...) and virtualenv doesn't help with that (except if you're crazy enough to use the --no-site-package option, and even then it's incomplete). The only solution I can think of for that situation is using different virtual machines.

So what am I missing? Can someone point me to a use case where PIP or virtualenv would be better than what I'm doing?

Answer 1

virtualenv really shines when you have a number of projects, and don't want them to all share the same Python installation. For example, you could have two project with conflicting requirements.