Django/Apache/mod_wsgi not using virtualenv's Python binary

Question

I have a virtualenv at /opt/webapps/ff/ with its own Python installation. I have WSGIPythonHome set to /opt/webapps/ff in my Apache config file (and this is definitely getting used in some capacity, because if I set it to a slightly different existing directory and restart Apache I get a 504). But if I e.g. assert False in a view somewhere to bring up the Django debug page, I see that settings.PYTHON_BIN is /usr/bin rather than /opt/webapps/ff/bin.

How do I get Apache/mod_wsgi to use my virtual environment's Python binary? I thought setting WSGIPythonHome was the way to do this, but it only seems to affect which site-packages dir is used, not which binary is used. Thanks.

Answer 1

These are the instructions I used which seem to be working well.

http://code.google.com/p/modwsgi/wiki/VirtualEnvironments

Using 'site.addsitedir()' is a bit different to simply adding the directory to 'sys.path' as the function will open up any '.pth' files located in the directory and process them. This is necessary to ensure that any special directories related to Python eggs are automatically added to 'sys.path'.

Note that although virtualenv includes the script 'activate_this.py', which the virtualenv documentation claims should be invoked using 'execfile()' in the context of mod_wsgi, you may want to be cautious using it. This is because the script modifies 'sys.prefix' which may actually cause problems with the operation of mod_wsgi or Python modules already loaded into the Python interpreter, if the code is dependent on the value of 'sys.prefix' not changing. The WSGIPythonHome directive already described should instead be used if wanting to associate Python as a whole with the virtual environment.

Despite that, the 'activate_this.py' script is an attempt to resolve an issue with how 'site.addsitedir()' works. That is that any new directories which are added to 'sys.path' by 'site.addsitedir()' are actually appended to the end. The problem with this in the context of mod_wsgi is that if WSGIPythonHome was not used to associate mod_wsgi with a virgin baseline environment, then any packages/modules in the main Python installation will still take precedence over those in the virtual environment.

To work around this problem, what 'activate_this.py' does is invoke 'site.addsitedir()' but then also reorders 'sys.path' so any newly added directories are shifted to the front of 'sys.path'. This will then ensure that where there are different versions of packages in the virtual environment that they take precedence over those in the main Python installation.

As explained, because 'activate_this.py' is doing other things which may not be appropriate in the context of mod_wsgi, if unable to set WSGIPythonHome to point mod_wsgi at a virgin baseline environment, instead of just calling 'site.addsitedir()' you should use the code:

ALLDIRS = ['usr/local/pythonenv/PYLONS-1/lib/python2.5/site-packages']

import sys 
import site 

# Remember original sys.path.
prev_sys_path = list(sys.path) 

# Add each new site-packages directory.
for directory in ALLDIRS:
  site.addsitedir(directory)

# Reorder sys.path so new directories at the front.
new_sys_path = [] 
for item in list(sys.path): 
    if item not in prev_sys_path: 
        new_sys_path.append(item) 
        sys.path.remove(item) 
sys.path[:0] = new_sys_path 

If you still want to use the activation script from virtualenv, then use:

activate_this = '/usr/local/pythonenv/PYLONS-1/bin/activate_this.py' 
execfile(activate_this, dict(__file__=activate_this))

If the fact that 'sys.prefix' has been modified doesn't give an issue, then great. If you see subtle unexplained problems that may be linked to the change to 'sys.prefix', then use the more long handed approach above whereby 'site.addsitedir()' is used directly and 'sys.path' reorderd subsequently.

Here is a discussion about this issue as well

http://groups.google.com/group/modwsgi/browse_thread/thread/466823f087070b5f?pli=1

Answer 2

I had the same situation in a Pylons app and ended up using /usr/bin binary plus virtualenv site-packages dir instead.

Of course it was the same python version...

Answer 3

If you're using a virtualenv, you need to be sure to activate it within the WSGI script.

venv_path = "/opt/webapps/ff"
activate_this = os.path.join(venv_path, "bin/activate_this.py")
execfile(activate_this, dict(__file__=activate_this))