Configuring Amazon Elastic Beanstalk with PostGIS

Question

Does anyone have any experience setting up Amazon Elastic Beanstalk with PostGIS (so that I can take advantage of Geodjango)?

There are a number of features that the default setup (RDS, featuring MySQL) does not currently support out of box: 1. PostgreSQL + PostGIS 2. The ability to install C/C++ libraries such as GEOS and Proj.4

Thanks in advance

Answer 1

If you want to use geodjango with Amazon Elastic Beanstalk you need to create a custom AMI where you can install PostGIS and then point your Elastic Beanstalk Application to that AMI when spinning up.

Here is a good tutorial on how to customize an EBS AMI. There is also an AWS tutorial for that but I found the first one easier to understand. On my custom AMI I installed geos, gdal, proj4 and postgis from source, and postgres using yum install postgres. Below are the commands i used to install all libraries into the AMI.

For the django app to find the libraries, I also set an additional environmental variable in the AWS EBS Console. In the menubar of my environment, I went to configuration --> software configuration and edited the Environment Properties by adding the property LD_LIBRARY_PATH set as /usr/local/lib/:$LD_LIBRARY_PATH.

Since the beanstalk app instances are not forseen to run the database themselves, I also set up a Amazon RDS Postgres hosted database which is a relatively new service, it supports PostGIS.

If you put that all together, you should get a very scaleable GeoDjango app!

sudo yum install postgresql postgresql-devel postgresql-server postgresql9-contrib gcc gcc-c++ make libtool curl libxml2 libxml2-devel python-devel

wget http://download.osgeo.org/proj/proj-4.8.0.zip
unzip proj-4.8.0.zip
cd proj-4.8.0
./configure
make
sudo make install
cd ..

wget http://download.osgeo.org/geos/geos-3.4.2.tar.bz2
tar -xvf geos-3.4.2.tar.bz2
cd geos-3.4.2
./configure
make
sudo make install
cd ..

wget http://download.osgeo.org/gdal/1.10.1/gdal1101.zip
unzip gdal1101.zip
cd gdal-1.10.1
./configure --with-python=yes
make
sudo make install
cd ..

wget http://download.osgeo.org/postgis/source/postgis-2.1.1.tar.gz
tar -xvf postgis-2.1.1.tar.gz
cd postgis-2.1.1
./configure
make
sudo make install

Answer 2

You can also do it without a custom AMI, just use ebextensions. I tested this with Amazon Instance (2013.09) ami-35792c5c so use that one instead of the newer ones. If you have your Django in Elastic Beanstalk 101 completed, you know about ebextensions. The ebextensions below will quickly get going you can use the following ebextensions. Just place the following in your .ebextensions folder at the base of your repository. I also include postgres 9.3 and memcached in these config files:

00_repo_ostgis.config:

files:
  "/etc/yum.repos.d/pgdg-93-redhat.repo":
    mode: "000644"
    owner: root
    group: root
    content: |
      [pgdg93]
      name=PostgreSQL 9.3 $releasever - $basearch
      baseurl=http://yum.postgresql.org/9.3/redhat/rhel-6-$basearch
      enabled=1
      gpgcheck=1
      gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-93

      [pgdg93-source]
      name=PostgreSQL 9.3 $releasever - $basearch - Source
      failovermethod=priority
      baseurl=http://yum.postgresql.org/srpms/9.3/redhat/rhel-6-$basearch
      enabled=0
      gpgcheck=1
      gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-93

commands:
  epel_repo:
    command: yum-config-manager -y --enable epel
  remi_repo:
    command: yum-config-manager -y --enable remi

packages:
  rpm:
    pgdg-redhat93-9.3-1: 'http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-redhat93-9.3-1.noarch.rpm'
    remi: 'http://rpms.famillecollet.com/enterprise/remi-release-6.rpm'
    qt4-devel: 'http://mirror.centos.org/centos/6/os/x86_64/Packages/qt-4.6.2-28.el6_5.x86_64.rpm'

01_app_postgis.config:

packages:
  yum:
    libtiff-devel: ''
    libjpeg-devel: ''
    libzip-devel: ''
    freetype-devel: ''
    postgresql-devel: ''
    gdal: ''
    gdal-python: ''
    geos: ''
    proj: ''
    libmemcached: ''
    libmemcached-devel: ''
    cyrus-sasl-devel: ''
    zlib-devel: ''

container_commands:
  01_collectstatic:
    command: 'PYTHONPATH=.:..:../lib cd site/<your_project> && ./manage.py collectstatic -c --noinput && cd ../..'
    leader_only: true
  02_migrate:
    command: 'PYTHONPATH=.:..:../lib cd site/<your_project> && ./manage.py migrate --noinput && cd ../..'
    leader_only: true

option_settings:
  - namespace: aws:elasticbeanstalk:container:python
    option_name: WSGIPath
    value: site/<your_project>/wsgi.py
  - namespace: aws:elasticbeanstalk:container:python:staticfiles
    option_name: /static/
    value: site/<your_project>/static/
  - option_name: DJANGO_SETTINGS_MODULE
    value: settings_prod

The structure of my project is a bit different. My settings file and urls.py I moved to the root of my project directory so i had to change the path to settings in wsgi.py. So adjust this accordingly. Just use the container_commands and option_settings you were using before.

Your requirements.txt file should contain at minimum:

Django==1.7.1
Pillow
psycopg2

I store most other python dependencies in ../lib which I include in my PYTHONPATH so my repo structure is like this:

<your_project>/
├── requirements.txt
├── .ebextensions/
│   ├── 00_repos_postgis.config
│   └── 01_app_postgis.config
└── site/
    ├── <your_project>
    │   ├── wsgi.py
    │   ├── settings_prod.py  # used for EB, like settings_local.py but uses env vars
    │   └── settings.py
    └── lib/
        └── <all pip dependencies>

Checkout the deployment tool I built, it uses fabric. I took what I liked from EB CLI tools and adjusted till it was tailored for django: https://github.com/radlws/django-awseb-tasks

NOTE: It is extremely important that you use AMI ami-35792c5c when you launch your environment. Its the only one that worked for me for this setup. If other instances work please feel free to edit them into this answer. Also see my original question.