Running Django migrations when deploying to Elastic Beanstalk

Question

I have my Django app set up on Elastic Beanstalk and recently made a change to the DB that I would like to have applied to the live DB now. I understand that I need to set this up as a container command, and after checking the DB I can see that the migration was run, but I can't figure out how to have more controls over the migration. For example, I only want a migration to run when necessary but from my understanding, the container will run the migration on every deploy assuming the command is still listed in the config file. Also, on occassion, I will be given options during a migration such as:

Any objects realted to these content types by a foreign key will also be deleted.
Are you sure you want to delete these content types?
If you're unsure, answer 'no'

How do I set up the container command to respond to this with a yes during the deployment phase?

This is my current config file

container_commands:
  01_migrate:
    command: 'source /opt/python/run/venv/bin/actiate && python app/manage.py makemigrations'
    command: 'source /opt/python/run/venv/bin/activate && python app/manage.py migrate'

Is there a way to set these 2 commands to only run when necessary and to respond to the yes/no options I receive during a migration?

Answer 1

The trick is that the full output of container_commands is in /var/log/cfn-init-cmd.log (Amazon Linux 2 Elastic Beanstalk released November 2020).

To view this you would run:

eb ssh [environment-name]
sudo tail -n 50 -f /var/log/cfn-init-cmd.log

This doesn't seem to be documented anywhere obvious and it's not displayed by eb logs; I found it by hunting around in /var/log.

The Django example management command django-admin.py migrate did not work for me. Instead I had to use something like:

01_migrate:
  command: "$PYTHONPATH/python manage.py migrate"
  leader_only: true
02_collectstatic:
  command: "$PYTHONPATH/python manage.py collectstatic --noinput --verbosity=0 --clear"

To see the values of your environment variables at deploy time, you can create a debug command like:

03_debug:
  command: "env"

You can see most of these environment variable with eb ssh; sudo cat /opt/elasticbeanstalk/deployment/env, but there seem to be some subtle differences at deploy time, hence using env above to be sure.

Here you'll see that $PYTHONPATH is being in a non-typical way, pointing to the virtualenv's bin directory, not the site-packages directory.

Answer 2

Aside from the automatic migration that you can add to deploy script (which runs every time you update the environment, and may not be desirable if you have long running migration or other Django management commands), you can ssh into an EB instance to run migration manually.

Here is how to manually run migration (and any other Django management commands) while working with Amazon Linux 2 (Python 3.7, 3.8) created by Elastic Beanstalk:

First, from your EB cli: eb ssh to connect an instance.

The virtual environment can be activated by source /var/app/venv/*/bin/activate

The manage.py can be ran by python3 /var/app/current/manage.py

Now the only tricky bit is to get Elastic Beanstalk's environment variables. You can access them by /opt/elasticbeanstalk/bin/get-config, I'm not super familiar with bash script, but here is a little script that I use to get and set environment variables, maybe someone can improve it to make it less hard-coded:

#! /bin/bash
export DJANGO_SECRET_KEY=$(/opt/elasticbeanstalk/bin/get-config environment -k DJANGO_SECRET_KEY)
...

More info regarding Amazon Linux 2 splatform script tools: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/custom-platforms-scripts.html