Docker Compose circular container linking

Question

I'm in the process of attempting to containerize our development environment with docker. This includes a mix of Google Appengine projects as well as services that are being eventually hosted in Google Compute engine within a containerized vm.

Our current development environment bootstrapping scripts bring up elasticsearch and nginx within boot2docker and the other applications run on localhost:{product port} within the dev_appserver appengine sandbox. This process is proving hard to manage and maintain as it takes a lot of insight into how our applications communicate.

I'm getting an error with docker-compose that is detecting a circular dependency between containers.

Circular import between cs and vbc and aa and sr.

As this configuration is only for development environments (mac osx), does anyone have an suggestions or ideas on a different approach to take when linking all of the product suites dependencies together.

A portion of docker-compose.yml:

elasticsearch:
  build: ./compute/containers/elasticsearch/elasticsearch
  ports:
    - "9200:9200"
  environment:
    - PROJECT_ID=localhost
nginx:
  build: ./compute/containers/elasticsearch/nginx
  links:
    - elasticsearch:localhost
  ports:
    - "9201:9201"
cs:
  build: ./CS
  command: dev_appserver.py /src/ --host=0.0.0.0 --admin_host=0.0.0.0 --port=8080 --admin_port=9080 --storage_path=/data/
  ports:
    - "8080:8080"
    - "9080:9080" 
  volumes:
   - /Users/source/CS/src:/src
   - /Users/source/CS/data:/data 
aa:
  build: ./AA
  command: dev_appserver.py /src/ --host=0.0.0.0 --admin_host=0.0.0.0 --port=8081 --admin_port=9081 --storage_path=/data/
  links:
    - vbc:vbc-local
    - st:st-local
    - elasticsearch:localhost    
  ports:
    - "8081:8081"
    - "9081:9081" 
  volumes:
   - /Users/source/AA/src:/src
   - /Users/source/AA/data:/data 
vbc:
  image: google/cloud-sdk
  command: dev_appserver.py /src/ --host=0.0.0.0 --admin_host=0.0.0.0 --port=8082 --admin_port=9082 --storage_path=/data/
  links:
    - cs:cs-local
    - sr:sr-local
    - sm:sm-local
    - ms:ms-local
    - st:st-local    
    - cis:cis-local
    - elasticsearch:localhost
  ports:
    - "8082:8082"
    - "9082:9082" 
  volumes:
   - /Users/source/VBC/src:/src
   - /Users/source/VBC/data:/data    
sr:
  build: ./SR
  command: dev_appserver.py /src/ --host=0.0.0.0 --admin_host=0.0.0.0 --port=8083 --admin_port=9083 --storage_path=/data/
  links:
    - cs:cs-local  
    - aa:aa-local      
  ports:
    - "8083:8083"
    - "9083:9083" 
  volumes:
   - /Users/source/SR/src:/src
   - /Users/source/SR/data:/data 

Answer 1

You should be able to use the following solution soon.

The circular linking is being fixed in PR # 1676

This is how they're addressing the issue. Simply put, they're going to make the containers able to speak to each other without linking. I've added the updates to the Docker Compose documentation below:

Networking in Compose

By default, Compose sets up a single default network for your app. Each container for a service joins the default network and is discoverable via DNS under the service's name.

Note: Your app's network is given the same name as the "project name", which is based on the name of the directory it lives in. See the CLI docs for how to override it.

For example, suppose your app is in a directory called myapp, and your docker-compose.yml looks like this:

web:
  build: .
  ports:
    - "8000:8000"
db:
  image: postgres

When you run docker-compose up, the following happens:

1. A network called myapp is created.
2. A container is created using web's configuration. It joins the network myapp under the name web.
3. A container is created using db's configuration. It joins the network myapp under the name db.

Each container can now look up the hostname web or db and get back the appropriate container's IP address. For example, web's application code could connect to the URL postgres://db:5432 and start using the Postgres database.

Because web explicitly maps a port, it's also accessible from the outside world via port 8000 on your Docker host's network interface.

Further reading on the experimental Docker networking API: https://github.com/docker/docker/blob/master/experimental/networking_api.md

Answer 2

Now with v2 docker-compose files definition, all the services are available between them without the need of a link section.

You can directly make a request to the service name from everyone to everyone (including a service to itself). So if you wanted to make a request from cs to vbc, you just curl vbc.

It's also possible to define a service with a custom domain name declaring a hostname key in the service section of the docker-compose file.

If you want to see more, the networking api is no longer experimental: https://github.com/docker/compose/blob/master/docs/networking.md

This is your docker-compose file in v2 without unnecessary links:

version: '2'

services:
  elasticsearch:
    build: ./compute/containers/elasticsearch/elasticsearch
    ports:
      - "9200:9200"
    environment:
      - PROJECT_ID=localhost
  nginx:
    build: ./compute/containers/elasticsearch/nginx
    ports:
      - "9201:9201"
  cs:
    build: ./CS
    command: dev_appserver.py /src/ --host=0.0.0.0 --admin_host=0.0.0.0 --port=8080 --admin_port=9080 --storage_path=/data/
    ports:
      - "8080:8080"
      - "9080:9080" 
    volumes:
     - /Users/source/CS/src:/src
     - /Users/source/CS/data:/data 
  aa:
    build: ./AA
    command: dev_appserver.py /src/ --host=0.0.0.0 --admin_host=0.0.0.0 --port=8081 --admin_port=9081 --storage_path=/data/
    ports:
      - "8081:8081"
      - "9081:9081" 
    volumes:
     - /Users/source/AA/src:/src
     - /Users/source/AA/data:/data 
  vbc:
    image: google/cloud-sdk
    command: dev_appserver.py /src/ --host=0.0.0.0 --admin_host=0.0.0.0 --port=8082 --admin_port=9082 --storage_path=/data/
    ports:
      - "8082:8082"
      - "9082:9082" 
    volumes:
     - /Users/source/VBC/src:/src
     - /Users/source/VBC/data:/data    
  sr:
    build: ./SR
    command: dev_appserver.py /src/ --host=0.0.0.0 --admin_host=0.0.0.0 --port=8083 --admin_port=9083 --storage_path=/data/
    ports:
      - "8083:8083"
      - "9083:9083" 
    volumes:
     - /Users/source/SR/src:/src
     - /Users/source/SR/data:/data