Connect to DynamoDB Local from inside docker container with boto3

Question

For testing, I am trying to run my python 3.4 application from inside docker, and connect to a DynamoDB Local instance. I can access DynamoDB local from the host without problems.

However, I get a "connection refused error" when hitting it from a docker container. I've tried running DynamoDB local in a container and linking it to the app's docker container, and running it on the host. One guess is that there are some environment variables missing, but I can't figure it out. I does not help to link my ~/.aws directory as a volume.

Here is some test code that reproduces the error:

import boto3

print('creating dynamodb resource')

dynamodb = boto3.resource(
    'dynamodb',
    endpoint_url='http://localhost:8001',
    region_name='dummy_region',
    aws_access_key_id='dummy_access_key',
    aws_secret_access_key='dummy_secret_key',
    verify=False)

print ('got resource:', dynamodb)

print('adding table')

result = dynamodb.create_table(
    TableName='foo',
        KeySchema=[
            {
                'AttributeName': 'from_email',
                'KeyType': 'HASH'  # Partition key
            },
            {
                'AttributeName': 'raw_id',
                'KeyType': 'RANGE'  # Sort key
            },
        ],
        AttributeDefinitions=[
            {
                'AttributeName': 'from_email',
                'AttributeType': 'S'
            },
            {
                'AttributeName': 'raw_id',
                'AttributeType': 'N'
            },
        ],
        ProvisionedThroughput={
            'ReadCapacityUnits': 10,
            'WriteCapacityUnits': 10
        }
    )

print('created table:', result)

print('getting table')

table = dynamodb.Table('foo')

print('got table:', table)

Running this from the host, here is my output:

(workbench) ryan@ryan:~/dev/$ python dyn.py 
creating dynamodb resource
got resource: dynamodb.ServiceResource()
adding table
created table: dynamodb.Table(name='foo')
getting table
got table: dynamodb.Table(name='foo')

Running the same code from inside the container shell:

root@e88da4d624e0:/src# python dyn.py 
creating dynamodb resource
got resource: dynamodb.ServiceResource()
adding table
[... traceback clipped ...]
  File "/usr/local/lib/python3.4/site-packages/botocore/vendored/requests/packages/urllib3/connection.py", line 155, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.4/site-packages/botocore/vendored/requests/packages/urllib3/connection.py", line 134, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.4/site-packages/botocore/vendored/requests/packages/urllib3/util/connection.py", line 88, in create_connection
    raise err
  File "/usr/local/lib/python3.4/site-packages/botocore/vendored/requests/packages/urllib3/util/connection.py", line 78, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
[ ... ]
botocore.vendored.requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))

Here is a test dockerfile that will reproduce the error:

FROM python:3.4

ADD . /src

# python dependencies
RUN python3 -m pip install -U pip && \
    python3 -m pip install boto3

ENTRYPOINT ["/bin/bash"]

And I am launching the docker containers like this:

docker run --name dynamodb -p "8001:8000" -d ryan/dynamodb
docker run --link dynamodb:localhost -ti ryan/app

Answer 1

You are using the docker link feature to connect two containers together. The fundamental principles here are:

1. You give your DB container a unique name, using --name.
2. You link your client container to the DB, using --link specifying the name of your DB container and an alias for it inside the client container.
3. Docker then connects the two using its default bridge network.

In doing so, it doesn't need a port mapping to the host network (as provided by the -p option). Instead docker will do the work for you and expose the IP addresses and ports as needed inside your client container, using the alias you defined when creating the link.

You can then use this alias to look up the details inside the client container using the environment variables docker automatically creates for you.

You've basically got it all right apart from this last step. If you check out your environment, I expect you'll find port 8000 is the one you want.