django cannot connect to RDS postgresql

Question

My Django application cannot connect to RDS PostgreSQL when it deployed to EC2. but oddly, it works fine when it running in my desktop.

EC2 server and desktop were configured with python3, django1.9, apache2, and mod_wsgi_py3

here is my settings.py database settings:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'postgres',
        'USER': 'some_name',
        'PASSWORD': 'some_password',
        'HOST': 'myhostname.cltlezrr85xn.ap-northeast-1.rds.amazonaws.com',
        'PORT': 5432,
    }
}

and error.log from apache2:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 199, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 171, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/postgresql/base.py", line 175, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/usr/local/lib/python3.4/dist-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
psycopg2.OperationalError: could not connect to server: Connection timed out
    Is the server running on host "myhostname.cltlezrr85xn.ap-northeast-1.rds.amazonaws.com" (172.--.--.---) and accepting
    TCP/IP connections on port 5432?


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.4/dist-packages/django/contrib/auth/management/commands/createsuperuser.py", line 52, in execute
    return super(Command, self).execute(*args, **options)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.4/dist-packages/django/contrib/auth/management/commands/createsuperuser.py", line 86, in handle
    default_username = get_default_username()
  File "/usr/local/lib/python3.4/dist-packages/django/contrib/auth/management/__init__.py", line 189, in get_default_username
    auth_app.User._default_manager.get(username=default_username)
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/manager.py", line 122, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 381, in get
    num = len(clone)
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 240, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 1074, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 52, in __iter__
    results = compiler.execute_sql()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/sql/compiler.py", line 846, in execute_sql
    cursor = self.connection.cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 231, in cursor
    cursor = self.make_debug_cursor(self._cursor())
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 204, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 199, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.4/dist-packages/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python3.4/dist-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 199, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 171, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/postgresql/base.py", line 175, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/usr/local/lib/python3.4/dist-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: could not connect to server: Connection timed out
    Is the server running on host "myhostname.cltlezrr85xn.ap-northeast-1.rds.amazonaws.com" (172.--.--.---) and accepting
    TCP/IP connections on port 5432?

thanks for suggestions..

Answer 1

Edit:

Just ran through this again, and have an easier way.

When you're trying to use an existing RDS database with Django on EC2 or EB, you'll have to adjust the security groups, and then get the proper parameters and set them as environment variables (RDS_*)

1) Create RDS, and match these up:

Environment variables - RDS console label

RDS_HOSTNAME - Endpoint (this is the hostname)
RDS_PORT - Port
RDS_DB_NAME –  DB Name
RDS_USERNAME –  Username
RDS_PASSWORD – Password you set for your DB

2) Set those using, for example, eb setenv

3) Go to your EC2/ EB instance and get the security group for that e.g. awseb-z-afsafdsaf-stack-AWSEBSecurityGroup-asfdsadfasdf

4) Go to the panel for your RDS instance, scroll down to Security Groups and take note of which security group it has. e.g. rds-launch-wizard-1 (ab-sdjfalkajsdf39)

5) Select the RDS security group, and add an Inbound rule with type: PostgreSQL (or whatever db you're using), and using the EC2 or EB instance that you got in step 2 as the source (awseb-z-afsafdsaf-stack-AWSEBSecurityGroup-asfdsadfasdf). Protocol and Port range should autopopulate.

6) Save it

That's it.

---

Original:

For anybody who stumbles upon this question:

When you're trying to use an existing RDS database with Django on EC2 or EB, you'll have to adjust the security groups, and then get the proper parameters and set them as environment variables (RDS_*)

1) Create RDS, and match these up:

Environment variables - RDS console label

RDS_HOSTNAME - Endpoint (this is the hostname)
RDS_PORT - Port
RDS_DB_NAME –  DB Name
RDS_USERNAME –  Username
RDS_PASSWORD – Password you set for your DB

2) Set those using, for example, eb setenv

3) Go to your EC2/ EB instance and get the security group for that e.g. awseb-z-afsafdsaf-stack-AWSEBSecurityGroup-asfdsadfasdf and for the load balancer: awseb-e-adsfadsf-stack-AWSEBLoadBalancerSecurityGroup-asdfadsf

4) Go to the panel for your RDS instance, scroll down to Security Groups and take note of which security group it has. e.g. rds-launch-wizard-1 (ab-sdjfalkajsdf39)

5) Click modify for the RDS instance, and in the Security Groups setting in the middle, add the load balancer security group you found above. It should have suggestions.

6) Go to the EC2 Dashboard and choose security groups from the menu on the left.

7) Select the load balancer security group, and add an Outbound rule. Type should be your RDS type (PostgreSQL) and the destination should be custom & the RDS instance's security group. Save.

8) Do the same for Inbound, use the same RDS type & Destination

9) Select the RDS security group, and add an Inbound rule, similar to 8, but using the EC2 or EB instance that you got in step 2.

10) Save, you're done. They should be able to work together now.

I'm not 100% certain all these steps are necessary, one or two might not be, but this got the job done for me.