virtualenv --system-site-packages not using system site packages

Question

I am under the impression that using the --system-site-packages flag with virtualenv will allow the virtual environment to use the already installed system packages. However I am finding that this is not the case. I am using a custom compiled version of python. You can see the problem in the steps below.

[user@machine django]$ which python
/app/python/bin/python
[user@machine django]$ which pip
/app/python/bin/pip
[user@machine django]$ which virtualenv
/app/python/bin/virtualenv
[user@machine django]$ python
Python 2.7.3 (default, Jul 27 2012, 11:30:41)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> exit()
[user@machine django]$ pip freeze
Django==1.4.2
distribute==0.6.30
mercurial==2.3.2
python-ldap==2.4.10
virtualenv==1.8.2
wsgiref==0.1.2
[user@machine django]$ pip --version
pip 1.2.1 from /app/python/lib/python2.7/site-packages (python 2.7)
[user@machine django]$ env
<snip>
LD_LIBRARY_PATH=/app/python/lib:/app/openldap/lib:/app/instantclient_11_2
PATH=/app/python/bin:/app/openldap/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/bin/cfdelivered:/home/user/bin:/app/oracle/product/java/jdk1.6.0_30/bin
PYTHONPATH=/app/python/lib/python2.7
[user@machine django]$ virtualenv --system-site-packages --distribute --python /app/python/bin/python2.7 foo
Running virtualenv with interpreter /app/python/bin/python2.7
New python executable in foo/bin/python2.7
Also creating executable in foo/bin/python
Installing distribute...<snip>...done.
Installing pip................done.
[user@machine django]$ . foo/bin/activate
(foo)[user@machine django]$ which python
/app/xxx/django/foo/bin/python
(foo)[user@machine django]$ which pip
/app/xxx/django/foo/bin/pip
(foo)[user@machine django]$ env
<snip>
LD_LIBRARY_PATH=/app/python/lib:/app/openldap/lib:/app/instantclient_11_2
VIRTUAL_ENV=/app/xxx/django/foo
PATH=/app/xxx/django/foo/bin:/app/python/bin:/app/openldap/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/bin/cfdelivered:/home/user/bin:/app/oracle/product/java/jdk1.6.0_30/bin
PYTHONPATH=/app/python/lib/python2.7
(foo)[user@machine django]$ python
Python 2.7.3 (default, Jul 27 2012, 11:30:41)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named django
>>>

Results of python2.7 -c "import sys;print(sys.path)"

Normal:

[user@machine django]$ python2.7 -c "import sys;print(sys.path)" ['', '/app/python/lib/python2.7/site-packages/cx_Oracle-5.1.2-py2.7-linux-x86_64.egg'‌​, '/app/python/lib/python2.7/site-packages/python_ldap-2.4.10-py2.7-linux-x86_64.e‌​gg', '/app/python/lib/python2.7', '/app/python/lib/python27.zip', '/app/python/lib/python2.7/plat-linux2', '/app/python/lib/python2.7/lib-tk', '/app/python/lib/python2.7/lib-old', '/app/python/lib/python2.7/lib-dynload', '/app/python/lib/python2.7/site-packages']

Inside venv:

(foo)[user@machine django]$ python2.7 -c "import sys;print(sys.path)" ['', '/app/xxx/django/foo/lib/python2.7/site-packages/distribute-0.6.28-py2.7.egg', '/app/xxx/django/foo/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg', '/app/python/lib/python2.7', '/app/xxx/django/foo/lib/python27.zip', '/app/xxx/django/foo/lib/python2.7', '/app/xxx/django/foo/lib/python2.7/plat-linux2', '/app/xxx/django/foo/lib/python2.7/lib-tk', '/app/xxx/django/foo/lib/python2.7/lib-old', '/app/xxx/django/foo/lib/python2.7/lib-dynload', '/app/xxx/django/foo/lib/python2.7/site-packages']

Answer 1

I'm not sure what went wrong when you created the virtualenv but clearly it does not have the expected values in its sys.path. If you use virtualenv x --system-site-packages to create virtual environment x, you should see the parent Python's site-packages directory on sys.path. In your listing above, the last item for the inside venv case should be the same as the normal case: /app/python/lib/python2.7/site-packages. You might try experimenting creating two simple virtualenv, with and without --system-site-packages, to see if that is the case. If not, you might try simplifying your configuration and trying again, like removing environment variables like PYTHONPATH which you should not need to set.