Django - Foreman cannot find installed modeles

Question

I am trying to use Foreman / Honcho to manage my Procfile-based Django application. When I start the app view the normal python manage.py runserver, everything works fine. However, when I start the app via honcho start or foreman start web, I am receiving this error:

11:59:31 system | web.1 started (pid=27959)
11:59:31 web.1  | [2016-04-26 11:59:31 -0700] [27959] [INFO] Starting gunicorn 19.4.5
11:59:31 web.1  | [2016-04-26 11:59:31 -0700] [27959] [INFO] Listening at: http://0.0.0.0:5000 (27959)
11:59:31 web.1  | [2016-04-26 11:59:31 -0700] [27959] [INFO] Using worker: sync
11:59:31 web.1  | [2016-04-26 11:59:31 -0700] [27962] [INFO] Booting worker with pid: 27962
11:59:31 web.1  | [2016-04-26 18:59:31 +0000] [27962] [ERROR] Exception in worker process:
11:59:31 web.1  | Traceback (most recent call last):
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/gunicorn/arbiter.py", line 515, in spawn_worker
11:59:31 web.1  |     worker.init_process()
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process
11:59:31 web.1  |     self.load_wsgi()
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/gunicorn/workers/base.py", line 130, in load_wsgi
11:59:31 web.1  |     self.wsgi = self.app.wsgi()
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
11:59:31 web.1  |     self.callable = self.load()
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
11:59:31 web.1  |     return self.load_wsgiapp()
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
11:59:31 web.1  |     return util.import_app(self.app_uri)
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/gunicorn/util.py", line 357, in import_app
11:59:31 web.1  |     __import__(module)
11:59:31 web.1  |   File "../wsgi.py", line 17, in <module>
11:59:31 web.1  |     application = get_wsgi_application()
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
11:59:31 web.1  |     django.setup()
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/django/__init__.py", line 18, in setup
11:59:31 web.1  |     apps.populate(settings.INSTALLED_APPS)
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/django/apps/registry.py", line 85, in populate
11:59:31 web.1  |     app_config = AppConfig.create(entry)
11:59:31 web.1  |   File "/Library/Python/2.7/site-packages/django/apps/config.py", line 90, in create
11:59:31 web.1  |     module = import_module(entry)
11:59:31 web.1  |   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
11:59:31 web.1  |     __import__(name)
11:59:31 web.1  | ImportError: No module named django_messages
11:59:31 web.1  | [2016-04-26 18:59:31 +0000] [27962] [INFO] Worker exiting (pid: 27962)
11:59:31 web.1  | [2016-04-26 11:59:31 -0700] [27959] [INFO] Shutting down: Master
11:59:31 web.1  | [2016-04-26 11:59:31 -0700] [27959] [INFO] Reason: Worker failed to boot.
11:59:31 system | web.1 stopped (rc=3)

This is with attempting to install the django-message module. I have the same issues with other modules as well. I'm also running into the same issue with django-webpack-loader. I should also mention that I am receiving the error both within a virtualenv and when it is deactivated.

Here's the command for installing django-messages:

$> pip install django-messages
Requirement already satisfied (use --upgrade to upgrade): django-messages in ./lib/python2.7/site-packages

Installed Apps;

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'my_app',
    'django_messages',
)

I'm not sure what other information I can provide to help troubleshoot, but the basic question is how do I get installed apps to work with foreman / honcho?

Answer 1

Honcho and Foreman don't use the Python executable and libs from your virtualenv, and while you didn't include your Honcho Procfile, just calling python will use the System-wide executable and libs.

Unfortunately, you can't just call /path/to/virtualenv/bin/activate as part of the Procfile, because Honcho exits when one of the subprocesses exits, as discussed in this Github issue thread. However, you can execute this script and your python script in one subshell using the && operator to chain them together:

web: source venv/bin/activate && python manage.py 

Alternatively, you might have better luck modifying your wsgi.py wrapper to explicitly pull in your virtualenv's libraries before importing your Django application:

# Activate your virtual env
activate_env=os.path.expanduser("/path/to/virtualenv/bin/activate_this.py")
execfile(activate_env, dict(__file__=activate_env))

These should be executed before importing any modules (other than os) to ensure that your application reads the correct site libraries.

Finally, Honcho itself supports the use of .env files alongside the Procfile which set up the environment the processes are run in. The format of this file is the same as any bash script. You could use the .env file to set PYTHONPATH and PYTHONHOME to point to the libraries in your Virtualenv, and then specify the explicit Python interpreter inside the Virtualenv from the Procfile.

.env File

PYTHONHOME=/path/to/virtualenv/lib/python2.7
PYTHONHOME=