BeautifulSoup inside Django view makes WSGI timeout

Question

For a strange reason when I instantiate a BeautifulSoup object inside Django's view, the WSGI timeout. Any help is appreciated as I am banging my head against the wall for hours and cannot find the root of this problem.

The view:

def index(request):
    soup = BeautifulSoup('<b>Bold</b>') # Removing this line solve the proble
    return HttpResponse('Hello')

The error message in Apache log:

[wsgi:error] [pid 4014] [client 127.0.0.1:50892] Timeout when reading response headers from daemon process 'test.local': /htdocs/test/test/wsgi.py

Update: This seems to be a bug in BeautifulSoup, however there is no soution.

Answer 1

Various third party packages for Python which use C extension modules, and this includes scipy, numpy and Beautifulsoup, will only work in the Python main interpreter and cannot be used in sub interpreters as mod_wsgi by default uses. You can find that in below link.

http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API

You can solve this by writing below line in your conf file.

WSGIApplicationGroup %{GLOBAL}

If running multiple WSGI applications on same server, you would want to start investigating using daemon mode because some frameworks don't allow multiple instances to run in same interpreter. This is the case with Django. Thus use daemon mode so each is in its own process and force each to run in main interpreter of their respective daemon mode process groups.