Python app import error in Django with WSGI gunicorn

Question

I'm trying to deploy a Django app with gunicorn on Heroku and I've run into a few hitches.

When I began my project my Django version was 1.3 and didn't contain the standard wsgi.py module, so I added the standard wsgi module as top/wsgi.py (top being my project name, turk being my app name, topturk being the containing directory - preserved so error logs make sense below).

Now when I run

gunicorn top.wsgi:application -b 0.0.0.0:$PORT

The server successfully starts up,

19:00:42 web.1     | started with pid 7869
19:00:42 web.1     | 2012-07-25 19:00:42 [7869] [INFO] Starting gunicorn 0.14.5
19:00:42 web.1     | 2012-07-25 19:00:42 [7869] [INFO] Listening at: http://0.0.0.0:5000 (7869)
19:00:42 web.1     | 2012-07-25 19:00:42 [7869] [INFO] Using worker: sync
19:00:42 web.1     | 2012-07-25 19:00:42 [7870] [INFO] Booting worker with pid: 7870

but then when I navigate to 0.0.0.0:5000 I get returned an Internal Server Error:

19:00:45 web.1     | 2012-07-25 17:00:45 [7870] [ERROR] Error handling request
19:00:45 web.1     | Traceback (most recent call last):
19:00:45 web.1     |   File "/Users/intenex/Dropbox/code/django/topturk/venv/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 102, in handle_request
19:00:45 web.1     |     respiter = self.wsgi(environ, resp.start_response)
19:00:45 web.1     |   File "/Users/intenex/Dropbox/code/django/topturk/venv/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 219, in __call__
19:00:45 web.1     |     self.load_middleware()
19:00:45 web.1     |   File "/Users/intenex/Dropbox/code/django/topturk/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 47, in load_middleware
19:00:45 web.1     |     raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
19:00:45 web.1     | ImproperlyConfigured: Error importing middleware turk.middleware.subdomain: "No module named turk.middleware.subdomain"
19:00:47 web.1     | 2012-07-25 17:00:47 [7870] [ERROR] Error handling request
19:00:47 web.1     | Traceback (most recent call last):
19:00:47 web.1     |   File "/Users/intenex/Dropbox/code/django/topturk/venv/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 102, in handle_request
19:00:47 web.1     |     respiter = self.wsgi(environ, resp.start_response)
19:00:47 web.1     |   File "/Users/intenex/Dropbox/code/django/topturk/venv/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 219, in __call__
19:00:47 web.1     |     self.load_middleware()
19:00:47 web.1     |   File "/Users/intenex/Dropbox/code/django/topturk/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 47, in load_middleware
19:00:47 web.1     |     raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
19:00:47 web.1     | ImproperlyConfigured: Error importing middleware turk.middleware.subdomain: "No module named turk.middleware.subdomain"

I'm assuming this is a python path error, where the server doesn't know how to import from my app directory

The relevant import code is here in settings:

MIDDLEWARE_CLASSES = (
    'turk.middleware.subdomain.SubdomainMiddleware',
    'turk.middleware.removewww.RemoveWWWMiddleware',
)

I attempted to fix this problem by inserting my app directory into sys.path like so at the top of my settings.py file like so:

PROJECT_ROOT = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(1, PROJECT_ROOT+'/turk/')

Which I've verified adds the app directory to the path, but still no dice. Any ideas? Also

sys.path.insert(1, PROJECT_ROOT+'/turk/')

seems hackish and adds at least two copies of the directory to the path, what's the correct way to append to PYTHON_PATH in Django? Thanks!

Answer 1

Figured out my problem. Needed to add the project directory to Python path, not the app directory - i.e., topturk/top instead of topturk/top/turk in order to import turk directory modules.

python top/manage.py run_gunicorn

and

python top/manage.py runserver

were working just fine because as per Python path documentation the directory of the calling module is always added as element 0 in the Python path tuple - and so when top/manage.py was being used, topturk/top was always in the Python path.

With heroku however, the Procfile is in the parent directory of the project, topturk and not topturk/top, so when the Procfile commands are run topturk is added to the Python path but not topturk/top, and hence the errors.

In hindsight, figured out this is what the Django documentation was referring to in the last sentence of this section: https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/gunicorn/#running-django-in-gunicorn-as-a-generic-wsgi-application, where they say in order to run this command the project must be on the Python path.

Problem solved by adding

sys.path.insert(1, os.path.dirname(os.path.realpath(__file__)))

to either settings.py or wsgi.py - added to settings.py as that seems like what some other people have recommended (http://codespatter.com/2009/04/10/how-to-add-locations-to-python-path-for-reusable-django-apps/), but not sure what the best place to put the insert is. Anyone know?

Answer 2

Well above method has been deprecated by gunicorn now. When I tried the same if failed with warning message!

(venv)root@ip-172-31-23-172:~/myproj# python manage.py run_gunicorn 0.0.0.0:8001

!!!

!!! WARNING: This command is deprecated.

!!!

!!! You should now run your application with the WSGI interface

!!! installed with your project. Ex.:

!!!

!!! gunicorn myproject.wsgi:application

!!!

!!! See https://docs.djangoproject.com/en/1.5/howto/deployment/wsgi/gunicorn/

!!! for more info.

!!!

Hence, I tried below command and it worked for me.

gunicorn myproject.wsgi:application