Problems with underscore in the domain name

Question

Let me first describe my original problem, and solution:

I have several docker-compose files which describe different parts of my application. The parts are developed and deployed independently, so they can not be integrated into a single compose file. But those components need to communicate with each other, and the solution I am using at the moment is to have an external network (bridge) which all services are connecting to. So far so good, and I can indeed connect to my service started with any docker compose file as long as I am connected to the custom bridge network:

$ docker run --network=mynet --rm --name ping_test -it xenial-networking bash

root@0319469f7951:/# ping -c 1 proj_web_1
PING proj_web_1 (172.30.0.3) 56(84) bytes of data.
64 bytes from 172.30.0.3: icmp_seq=1 ttl=64 time=0.071 ms

--- proj_web_1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.071/0.071/0.071/0.000 ms

Why am using proj_web_1? Because that is the DNS entry that docker compose is creating.

This works fine, and nobody cares that underscores are not that great in domain names.

Nobody ... except django it seems:

ERROR Invalid HTTP_HOST header: 'proj_web_1:8000'. The domain name provided is not valid according to RFC 1034/1035.
172.30.0.5 - admin [07/May/2018:05:41:53 +0000] "OPTIONS /api/v1/expansions/ HTTP/1.1" 400 58663 "-" "python-requests/2.18.4"

It seems that docker-compose does not support using hyphens instead of underscores.

It could be that I can workaround this by enabling DEBUG in my django service.

Is there a cleaner way of enabling this, without running in django debug mode? Is there a way to tell docker compose to stop using underscores?

Answer 1

Cross-compose network connections

When running a compose project, services are accessible both through their full name (including the project-name prefix, for example, myproject_web_1), and through their service name (as specified in the compose-file), for example web. The short name is a network-scoped alias, which means that any container connected to the same network can access the container through this name.

By default, docker-compose creates a network for each compose project (projectname_default) so that all services in the compose project can communicate. Because that network is created for each project individually, two compose project don't share the same network, and their services are isolated from other compose projects.

It's possible, however, to make compose projects (or individual services in a compose project) share the same network.

Example 1 - using a shared network for the whole project

The following compose files specify a custom name for the default network; both compose-files use the sharednet network as the default, which means that services for both compose project will be connected to the same network:

Compose file 1 (compose1.yml):

version: '3.5'
services:
  compose1service:
    image: busybox
    tty: true

networks:
  default:
    name: sharednet

Compose file 2 (compose2.yml):

version: '3.5'
services:
  compose2service:
    image: busybox
    tty: true

networks:
  default:
    name: sharednet

To illustrate this:

Start both compose files:

docker-compose -f compose1.yml --project-name=compose1 up -d
docker-compose -f compose2.yml --project-name=compose2 up -d

Using the short (compose2service) name to ping the compose2service from inside the service in the compose1service container works;

docker exec compose1_compose1service_1 ping -c1 compose2service
PING compose2service (172.20.0.3): 56 data bytes
64 bytes from 172.20.0.3: seq=0 ttl=64 time=0.134 ms

--- compose2service ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.134/0.134/0.134 ms

And vice-versa:

docker exec compose2_compose2service_1 ping -c1 compose1service
PING compose1service (172.20.0.2): 56 data bytes
64 bytes from 172.20.0.2: seq=0 ttl=64 time=0.151 ms

--- compose1service ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.151/0.151/0.151 ms

As does the full name (including the project prefix);

docker exec compose2_compose2service_1 ping -c1 compose1_compose1service_1
PING compose1_compose1service_1 (172.20.0.2): 56 data bytes
64 bytes from 172.20.0.2: seq=0 ttl=64 time=0.151 ms

--- compose1_compose1service_1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.151/0.151/0.151 ms

Example 2 - using a shared network for some service in the project

In this example, both compose projects have their own ("default") network, but in addition, a shared network is added to allow some services to connect to services in the other compose project.

Compose file 1 (compose3.yml):

version: '3.5'
services:
  compose3service:
    image: busybox
    tty: true
    networks:
      - default
      - sharednet

  compose3otherservice:
    image: busybox
    tty: true

networks:
  sharednet:
    name: mysharednetwork

Note: if you specify which networks a service should be connected to, you override the defaults, which means that the service is no longer automatically connected to the default network. Include the default network in the list of networks to allow the service to communicate with other services in the compose project.

Compose file 2 (compose4.yml):

version: '3.5'
services:
  compose4service:
    image: busybox
    tty: true
    networks:
      - default
      - sharednet

  compose4otherservice:
    image: busybox
    tty: true

networks:
  sharednet:
    name: mysharednetwork

Start both compose files:

docker-compose -f compose3.yml --project-name=compose3 up -d
docker-compose -f compose4.yml --project-name=compose4 up -d

Again, pinging from the compose3service to the compose4service service in the other compose project works (either by short name or full name);

docker exec compose3_compose3service_1 ping -c1 compose4service
PING compose4service (172.22.0.3): 56 data bytes
64 bytes from 172.22.0.3: seq=0 ttl=64 time=0.110 ms

--- compose4service ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.110/0.110/0.110 ms


docker exec compose3_compose3service_1 ping -c1 compose4_compose4service_1
PING compose4_compose4service_1 (172.22.0.3): 56 data bytes
64 bytes from 172.22.0.3: seq=0 ttl=64 time=0.093 ms

--- compose4_compose4service_1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.093/0.093/0.093 ms

However, trying the same to connect to the compose4otherservice will fail, because that service is not connected to the mysharednetwork network:

docker exec compose3_compose3service_1 ping -c1 compose4otherservice
ping: bad address 'compose4otherservice'

docker exec compose3_compose3service_1 ping -c1 compose4_compose4otherservice_1
ping: bad address 'compose4_compose4otherservice_1'

Note:

Care should be taken if two compose projects have a service with the same name (for example, both have a service named web). In such situations, web may at random resolve to the web service from either project.