docker-compose: connection refused between containers, but service accessible from host

Question

TL;DR: How do I have to change my below docker-compose.yml in order to allow one container to use a service of another over a custom (non-standard) port?

I have a pretty common setup: containers for a web app (Padrino [Ruby]), Postgres, Redis, and a queueing framework (Sidekiq). The web app comes with its custom Dockerfile, the remaining services come either from standard images (Postgres, Redis), or mount the data from the web app (Sidekiq). They are ties together via the following docker-compose.yml:

version: '2'

services:
  web:
    build: .
    command: 'bundle exec puma -C config/puma.rb'
    volumes:
      - .:/myapp
    ports:
      - "9000:3000"
    depends_on:
      - postgres
      - redis

  sidekiq:
    build: .
    command: 'bundle exec sidekiq -C config/sidekiq.yml -r ./config/boot.rb'
    volumes:
      - .:/myapp
    depends_on:
      - postgres
      - redis

  postgres:
    image: postgres:9.5
    environment:
      POSTGRES_USER: my-postgres-user
      POSTGRES_PASSWORD: my-postgres-pass
    ports:
      - '9001:5432'
    volumes:
      - 'postgres:/var/lib/postgresql/data'

  redis:
    image: redis
    ports:
      - '9002:6379'
    volumes:
      - 'redis:/var/lib/redis/data'

volumes:
  redis:
  postgres:

One key point to notice here is that I am exposing the containers services on non-standard ports (9000-9002).

If I start the setup with docker-compose up, the Redis and Postgres containers come up fine, but the containers for the web app and Sidekiq fail since they can't connect to Redis at redis:9002. Remarkably enough, the same setup works if I use 6379 (the standard Redis port) instead of 9002.

docker ps also looks fine afaik:

CONTAINER ID        IMAGE                   COMMAND                  CREATED                  STATUS              PORTS                              NAMES
9148566c2509        redis                   "docker-entrypoint.sh"   Less than a second ago   Up About a minute   0.0.0.0:9002->6379/tcp             rubydockerpadrino_redis_1
e6d47321c939        postgres:9.5            "/docker-entrypoint.s"   Less than a second ago   Up About a minute   0.0.0.0:9001->5432/tcp             rubydockerpadrino_postgres_1

What's even more confusing: I can access the Redis container from the host via redis-cli -h localhost -p 9002 -n 0, but the web app and Sidekiq containers fail to establish a connection.

I am using this docker version on MacOS: Docker version 1.12.3, build 6b644ec, experimental

Any ideas what I am doing wrong? I'd appreciate any hint how to get my setup running.

Answer 1

When you bind ports like this '9002:6379' you're telling Docker to forward traffic from localhost:9002 -> redis:6379. That's why this works from your host machine:

redis-cli -h localhost -p 9002 -n 0

However, when containers talk to each other, they are all connected to the same network by default (the Docker bridge or docker0). By default, containers can communicate with each other freely on this network, without needing any ports opened. Within this network, your redis container is listening for traffic on it's usual port (6379), host isn't involved at all. That's why your container to container communication works on 6379.