Share Python Interpreter in Apache Prefork / WSGI

Question

I am attempting to run a Python application within Apache (prefork) with WSGI in such a way that a single Python interpreter will be used. This is necessary since the application uses thread synchronization to prevent race conditions from occurring. Since Apache prefork spawns multiple processes, the code winds up not being shared between the interpreters and thus the thread synchronization is irrelevant (i.e. each thread only sees it own locks which have no bearing on the other processes).

Here is the setup:

• Apache 2.0 (prefork)
• WSGI
• Python 2.5

Here is the relevant Apache configuration:

WSGIApplicationGroup %{GLOBAL}
<VirtualHost _default_:80>

WSGIScriptAlias / /var/convergedsecurity/apache/osvm.wsgi

Alias /admin_media/ /var/www/html/admin_media/

<Directory /var/www/html/admin_media>
Order deny,allow
Allow from all
</Directory>

Alias /media/ /var/www/html/media/

<Directory /var/www/html/media>
Order deny,allow
Allow from all
</Directory>

</VirtualHost>

Here is what I tried so far (none of which worked):

1. Adding WSGIApplicationGroup %{GLOBAL}

2. Specifying WSGIDaemonProcess and WSGIProcessGroup within the virtual host:

   WSGIDaemonProcess osvm threads=50
   WSGIProcessGroup osvm

Is there no way to force Apache prefork to use a single Python interpreter with WSGI? The documents seem to imply you can with the WSGIDaemonProcess and WSGIApplicationGroup options but Apache still creates a separate Python interpreter for each process.

Answer 1

You can't have the WSGI application run in embedded mode on UNIX systems, whether it be prefork or worker MPM, as there will indeed be multiple processes. See:

http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading

Creating a daemon process group consisting of single process and delegating WSGI application to that should achieve what you want. You shouldn't even need to use WSGIApplicationGroup if it is only one mounted WSGI application you are talking about. If you want to be absolutely sure though, you can also set it.

Thus configuration within VirtualHost would be:

WSGIDaemonProcess osvm
WSGIProcessGroup osvm
WSGIApplicationGroup %{GLOBAL}

WSGIScriptAlias / /var/convergedsecurity/apache/osvm.wsgi

Although 'processes=1' for WSGIDaemonProcess makes it explicit that one process is created, don't provide the option though and just let it default to one process. Any use of 'processes' option, even if for one process will see 'wsgi.multiprocess' set to True.

Rather than use your actual WSGI application, I would suggest you test with the following simple test program.

import cStringIO
import os

def application(environ, start_response):
    headers = []
    headers.append(('Content-Type', 'text/plain'))
    write = start_response('200 OK', headers)

    input = environ['wsgi.input']
    output = cStringIO.StringIO()

    print >> output, "PID: %s" % os.getpid()
    print >> output

    keys = environ.keys()
    keys.sort()
    for key in keys:
        print >> output, '%s: %s' % (key, repr(environ[key]))
    print >> output

    output.write(input.read(int(environ.get('CONTENT_LENGTH', '0'))))

    return [output.getvalue()]

In the output of that, the PID value should always be the same. The wsgi.multiprocess flag should be False. The mod_wsgi.process_group value should be what ever you called the daemon process group. And the mod_wsgi.application_group should be an empty string.

If this isn't what you are seeing, ensure you actually restarted Apache after making configuration changes. Also add:

LogLevel debug

to Apache configuration for VirtualHost. Doing that will cause mod_wsgi to log a lot more messages in Apache error log about process creation and script loading, including details of process group and application group things are happening for.

For other information on debugging, see:

http://code.google.com/p/modwsgi/wiki/DebuggingTechniques

If still problems, suggest you go to the mod_wsgi mailing list on Google Groups.