Container Command Fails in Django on Elastic Beanstalk Python 3.7

Question

I am using Django Python 3.7 on Elastic Beanstalk (Amazon Linux 2) and the following command fails:

container_commands:
  01_migrate:
    command: "pipenv run python ./manage.py migrate"
    leader_only: true

2020-07-17 09:31:57,017 [ERROR] Command 01_migrate (pipenv run python ./manage.py migrate) failed
2020-07-17 09:31:57,017 [ERROR] Error encountered during build of postbuild_0_sarahandanatol: Command 01_migrate failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 542, in run_config
    CloudFormationCarpenter(config, self._auth_config).build(worklog)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 260, in build
    changes['commands'] = CommandTool().apply(self._config.commands)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/command_tool.py", line 117, in apply
    raise ToolError(u"Command %s failed" % name)
ToolError: Command 01_migrate failed

Answer 1

I tried to replicate the issue on my sandbox account. It was minimal version of the django just with the welcome screen. No database nor the use of environmental variables.

My Pipfile was minimal as well:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
django = "*"
djangorestframework = "*"

[requires]
python_version = "3.7"

I can confirm that using plain pipenv fails. Specifically, I had the following config file in my .ebextantions

container_commands:
  10_migrate:
    command: |
      pipenv run python ./manage.py migrate

The error message was about missing Django.

However, for me the solution was the following:

container_commands:
  10_migrate:
    command: |      
      source $PYTHONPATH/activate
      pipenv run python ./manage.py migrate

This activates python environment that EB is using to install Pipfile dependencies before pipenv is executed.

Below is a version which will also load EB environmental variables that maybe required to run the migrate job if you have database connection details passed as such.

container_commands:
  10_migrate:
    command: |
      export $(cat /opt/elasticbeanstalk/deployment/env | xargs)
      source $PYTHONPATH/activate
      pipenv run python ./manage.py migrate

Here is an example output from /var/log/cfn-init-cmd.log showing successful migrate run:

20-07-18 04:50:41,615 P3836 [INFO] Command 10_migrate
2020-07-18 04:50:42,969 P3836 [INFO] -----------------------Command Output-----------------------
2020-07-18 04:50:42,969 P3836 [INFO]    cat: /opt/elasticbeanstalk/deployment/env: No such file or directory
2020-07-18 04:50:42,969 P3836 [INFO]    export EB_IS_COMMAND_LEADER="true"
2020-07-18 04:50:42,969 P3836 [INFO]    export HOME="/root"
2020-07-18 04:50:42,969 P3836 [INFO]    export MYVAR="my-eb-env-value"
2020-07-18 04:50:42,969 P3836 [INFO]    export OLDPWD
2020-07-18 04:50:42,970 P3836 [INFO]    export PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"
2020-07-18 04:50:42,970 P3836 [INFO]    export PWD="/var/app/staging"
2020-07-18 04:50:42,970 P3836 [INFO]    export PYTHONPATH="/var/app/venv/staging-LQM1lest/bin"
2020-07-18 04:50:42,970 P3836 [INFO]    export SHLVL="4"
2020-07-18 04:50:42,970 P3836 [INFO]    export _="/bin/jq"
2020-07-18 04:50:42,970 P3836 [INFO]    Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
2020-07-18 04:50:42,970 P3836 [INFO]    Operations to perform:
2020-07-18 04:50:42,970 P3836 [INFO]      Apply all migrations: admin, auth, contenttypes, sessions
2020-07-18 04:50:42,970 P3836 [INFO]    Running migrations:
2020-07-18 04:50:42,970 P3836 [INFO]      Applying contenttypes.0001_initial... OK
2020-07-18 04:50:42,970 P3836 [INFO]      Applying auth.0001_initial... OK
2020-07-18 04:50:42,970 P3836 [INFO]      Applying admin.0001_initial... OK
2020-07-18 04:50:42,970 P3836 [INFO]      Applying admin.0002_logentry_remove_auto_add... OK
2020-07-18 04:50:42,970 P3836 [INFO]      Applying admin.0003_logentry_add_action_flag_choices... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying contenttypes.0002_remove_content_type_name... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0002_alter_permission_name_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0003_alter_user_email_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0004_alter_user_username_opts... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0005_alter_user_last_login_null... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0006_require_contenttypes_0002... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0007_alter_validators_add_error_messages... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0008_alter_user_username_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0009_alter_user_last_name_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0010_alter_group_name_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0011_update_proxy_permissions... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying sessions.0001_initial... OK