Python virtualenv is very slow

Question

I'm working on Ubuntu 12.04 x64 with Python 2.7.3 and I've just ported a Django 1.5.4 site to virtualenv 1.7.1.2.

However, I've noticed that Python commands are much slower when they are run from an active virtualenv than when using the system's Python installation. E.g. runnig python manage.py validate with system's Python lasts ~1s while the same command run inside a virtualenv environment lasts ~4s. Is this normal? Am I missing something? Will I have efficiency issues when deploying this site with Apache + virtualenv?

Note: When the local Django development server is running I haven't noticed efficiency issues while using the site. It seems to only affect python commands run from shell inside a virtualenv... Is this true?

Update1:

Here you have an example of the times I'm actually getting:

enric@developer:~/Documentos/workspace/EurekaStart$ time python manage.py validate
0 errors found

real    0m1.049s
user    0m0.648s
sys     0m0.120s
enric@developer:~/Documentos/workspace/EurekaStart$ source env/bin/activate
(env)enric@developer:~/Documentos/workspace/EurekaStart$ time python manage.py validate
0 errors found

real    0m5.261s
user    0m0.968s
sys     0m1.032s

Update 2:

I've done further testing and updated virtualenv to the latest version 1.10.1 and I've noticed that the times times have improved, but just a litte (between 0.5s and 1s). I have also tried to create the env with the --system-site-packages flag and time is much better, but still slower than using system's default Python. Here are the times:

Using env created without using system-site-packages:

(env)enric@developer:~/Documentos/workspace/EurekaStart$ time python manage.py validate
0 errors found

real    0m4.648s
user    0m1.008s
sys     0m0.824s

Using env2 created using system-site-packages:

(env2)enric@developer:~/Documentos/workspace/EurekaStart$ time python manage.py validate
0 errors found

real    0m1.921s
user    0m0.760s
sys     0m0.312s

Just to note, normal execution without ANY env lasts 1s (as stated in update1).

---

I've just created the same virtualenv with exactly the same libs installed inside from a fresh Ubuntu 12.04 (live CD) and everything works as expected! It even ran faster on this new machine: 0.6s instead of 1.0 in my machine.

One interesting point to note: If I use the virtualenv created from the fresh Ubuntu in my real machine it runs as fast as executing the command without virtualenv, which is what is actually expected. But, if I use the virtualenvs created from this machine, then they run very slow.

So, I guess that the libs installed in the system really influence on how a virtualenv is created.

Now, the question would be... Is there any way to purge my Ubuntu installation to be able to create efficient virtualenvs? Is there any known lib that may cause this issue? (There aren't broken packages in the system, as I've already checked this).

Answer 1

This applies primarily to Vagrant-powered virtualenvs, but I ran into the same problem: extremely slow manage.py / django-admin.py (~ 10-15s. for django-admin.py version).

I found the culprit to be vagrant/virtualbox, and specifically generating the virtualenv on a shared folder.

Regenerating the virtualenv on a non-shared folder (like ~/.envs) solved the problem for me.

Hope it helps someone.

Answer 2

I think that the efficiency issues with virtualenv may be caused by the type of HDD partition it is allocated. In my case, running virtualenv from a NTFS partition causes it to be really slow. However, running a virtualenv with exactly the same libs from an Ext4 partition works as expected.