Invalid requirements.txt on deploying django app to aws beanstalk

Question

I have been deploying my app without any issues to the beanstalk environment until my latest commit.

Now I get this

Time    Type    Details 2013-09-01 10:19:12 UTC+0800    ERROR   Failed to deploy application. 2013-09-01 10:19:11 UTC+0800    ERROR   Responses from [i-50f40d06] were received, but the commands failed. 2013-09-01 10:19:11 UTC+0800    ERROR   Your requirements.txt is invalid. Snapshot your logs for details. 2013-09-01 10:19:11 UTC+0800    ERROR   [Instance: i-50f40d06 Module: AWSEBAutoScalingGroup ConfigSet: Hook-PreAppDeploy] Failed on instance with return code: 1 Output: Error occurred during build: Command hooks failed . 

The snapshot logs say this:

2013-09-01 02:19:08,852 [INFO] (9941 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Output from script: New python executable in /opt/python/run/venv/bin/python2.6 Not overwriting existing python script /opt/python/run/venv/bin/python (you must use /opt/python/run/venv/bin/python2.6) Installing distribute..................................................................................................................................................................................................done. Installing pip................done. 2013-09-01 02:19:08,564 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install --use-mirrors -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status -9 Traceback (most recent call last):   File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 31, in main     install_dependencies()   File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 25, in install_dependencies     shell=True)   File "/usr/lib/python2.6/subprocess.py", line 502, in check_call     raise CalledProcessError(retcode, cmd) CalledProcessError: Command '/opt/python/run/venv/bin/pip install --use-mirrors -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status -9  2013-09-01 02:19:08,853 [ERROR] (9941 MainThread) [directoryHooksExecutor.py-34] [root directoryHooksExecutor error] Script failed with returncode 1 2 

My requirements.txt is the same as before. I tried removing everything in the requirements.txt as well but I get the same error again which makes me think the problem was before requirements.txt was run.

I don't know the order in which things are deployed. ANyway, here is my config file:

packages:   yum:     libjpeg-devel: '6b'     zlib-devel: []     freetype-devel: []   container_commands:   01_install_mysqldb:     command: "pip install distribute==0.6.28; pip install mysql-python;"   02_syncdb:     command: "python manage.py syncdb --noinput"     leader_only: true   03_createadmin:     command: "python scripts/createadmin.py"     leader_only: true   04_collectstatic:     command: "python manage.py collectstatic --noinput"   05_migrate_wizards:     command: "python manage.py migrate wizards --noinput"     leader_only: true   06_migrate_facebook:     command: "python manage.py migrate facebook --noinput"     leader_only: true   07_migrate_socialaccount:     command: "python manage.py migrate socialaccount 0011 --noinput"     leader_only: true   08_migrate_missions:     command: "python manage.py migrate missions --noinput"     leader_only: true   09_migrate_mailchimp:     command: "python manage.py migrate mailchimp --noinput"     leader_only: true   10_migrate_actstream:     command: "python manage.py migrate actstream --noinput"     leader_only: true  option_settings:   - namespace: aws:elasticbeanstalk:container:python     option_name: WSGIPath     value: ideatory/wsgi.py   - namespace: aws:elasticbeanstalk:container:python:staticfiles     option_name: /static/     value: static/   - option_name: DJANGO_SETTINGS_MODULE     value: ideatory.settings 

Answer 1

I had the exact same problem. The only thing that worked for me was (automatically) rebuilding my entire environment. You can do that in the AWS console. Go to your environment and click Actions > Rebuild Environment. This will take some time, afterwards it will automatically relaunch your app without this error.

Update:

The problem keeps returning so now and then. I figured out that something is going wrong when pip compiles psycopg, which is needed for postgreSQL support. Haven't found a real solution yet.

Answer 2

I was able to make this work by adding postgresql93-devel and postgresql93 to my .ebextensions

as described in Customizing the Software on EC2 Instances Running Linux

This is working for 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7

in .ebextensions/01_pre_deps.config:

packages:   yum:     gcc-c++: []     make: []     openssl-devel: []     git: []     python-devel: []     python27-devel: []     ruby-devel: []     postgresql93: []     postgresql93-devel: []     # nginx: []   rubygems:     bundler: '1.7.3'     compass-core: '1.1.0.alpha.3'     neat-compass: '1.6.0'