ZeroMQ failing to communicate between two Docker containers

Question

I'm trying to set up a toy example of Docker networking with ZeroMQ in macOS, where the serverd.py sends a message to the clientd.py and the client simply displays it using PUSH/PULL. If I run them outside of the container they work fine, but I'm having trouble getting them to communicate when running within separate containers. It seems like my clientd.py cannot connect to the container name despite them being within the same bridge network. I tried replacing the hostname with the assigned ip address for serverd_dev_1, but this doesn't work either.

Here's my setup:

1. I created a new network with docker network create -d bridge mynet. Here is the output from docker network inpsect mynet:

   {
       "Name": "mynet",
       "Id": "cec7f8037c0ef173d9a9a66065bb46cb6a631fea1c0636876ccfe5a792f92412",
       "Created": "2017-08-19T09:52:44.8034344Z",
       "Scope": "local",
       "Driver": "bridge",
       "EnableIPv6": false,
       "IPAM": {
           "Driver": "default",
           "Options": {},
           "Config": [
               {
                   "Subnet": "172.18.0.0/16",
                   "Gateway": "172.18.0.1"
               }
           ]
       },
       "Internal": false,
       "Attachable": false,
       "Ingress": false,
       "ConfigFrom": {
           "Network": ""
       },
       "ConfigOnly": false,
       "Containers": {
           "5fa8dc2f8059d675dfd3dc4f2e50265be99361cd8a8f2730eb273772c0148742": {
               "Name": "serverd_dev_1",
               "EndpointID": "3a62e82b1b34d5c08f2a9f340ff93aebd65c0f3dfde70e354819befe21422d0b",
               "MacAddress": "02:42:ac:12:00:02",
               "IPv4Address": "172.18.0.2/16",
               "IPv6Address": ""
           },
           "ec1e5f8c525ca8297611e02bcd3a64198fda3a07ce8ed82c0c4298609ba0357f": {
               "Name": "clientd_dev_1",
               "EndpointID": "a8ce6f178a225cb2d39ac0009e16c39abdd2dae02a65ba5fd073b7900f059bb8",
               "MacAddress": "02:42:ac:12:00:03",
               "IPv4Address": "172.18.0.3/16",
               "IPv6Address": ""
           }
       },
       "Options": {},
       "Labels": {}
   }
   

2. I created serverd.py and clientd.py like this and put them in separate folders along with their Dockerfiles and docker-compose.yml:

serverd.py:

import zmq
import time

context = zmq.Context()
socket = context.socket(zmq.PUSH)
address = "tcp://127.0.0.1:5557"
socket.bind(address)
print("Sending to {}...".format(address))
while True:
    message = socket.send_string("Got it!")
    print("Sent message")
    time.sleep(1)

clientd.py:

import zmq

context = zmq.Context()
socket = context.socket(zmq.PULL)
address = "tcp://serverd_dev_1:5557"
socket.connect(address)
print("Listening to {}...".format(address))
while True:
    message = socket.recv_string()
    print("Client got message! {}".format(message))

I have two Dockerfiles and docker-compose.yml:

Dockerfile for serverd.py:

FROM python:3.6

RUN mkdir src
ADD serverd.py /src/
RUN pip install pyzmq
WORKDIR /src/
EXPOSE 5557

Dockerfile for clientd.py:

FROM python:3.6

RUN mkdir src
ADD clientd.py /src/
RUN pip install pyzmq
WORKDIR /src/
EXPOSE 5557

docker-compose.yml for serverd.py:

dev:
  build: .
  command: ["python", "-u", "./serverd.py"]
  net: mynet

docker-compose for clientd.py:

dev:
  build: .
  command: ["python", "-u", "./clientd.py"]
  net: mynet

3. serverd.py starts up as expected with docker-compose up:

Sending to tcp://127.0.0.1:5557...

4. clientd.py won't start up like this due to it not finding the hostname tcp://serverd_dev_1:5557.

   Attaching to countd_dev_1
   dev_1  | Traceback (most recent call last):
   dev_1  |   File "./countd.py", line 6, in <module>
   dev_1  |     socket.connect(address)
   dev_1  |   File "zmq/backend/cython/socket.pyx", line 528, in zmq.backend.cython.socket.Socket.connect (zmq/backend/cython/socket.c:5971)
   dev_1  |   File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:10014)
   dev_1  | zmq.error.ZMQError: Invalid argument
   

5. If I replace the URI tcp://serverd_dev_1:5557 with tcp://172.18.0.2:5557 it does not crash anymore, but it is simply idling without receiving any of the messages from the server. Obviously I'm doing something wrong, but I'm not entirely sure what exactly. I feel like I have been following the Docker documentation as closely as possible, and would very much be grateful if you had any ideas.

Answer 1

Your primary problem is that you have configured your server with the address tcp://127.0.0.1:5557. Because it's bound to localhost (127.0.0.1), that socket isn't going to be visible to anything outside of that container. So the first thing you need to fix is the server bind address. Consider:

address = "tcp://0.0.0.0:5557"

A second problem is that you're using the name serverd_dev_1 in the client, but it's not clear this would actually be the name of your serverd container(that would depend on the directory names in use when you run docker-compose up).

Naming is easier to manage with a single docker-compose.yaml file. For example, I set things up like this:

version: "2"

services:
  serverd:
    build: serverd
    command: ["python", "-u", "./serverd.py"]
    environment:
      SERVER_LISTEN_URI: tcp://0.0.0.0:5557

  clientd:
    build: clientd
    command: ["python", "-u", "./clientd.py"]
    environment:
      SERVER_CONNECT_URI: tcp://serverd:5557

This will launch both containers in a dedicated network (because this is what docker-compose does by default), so you don't need to explicitly create or reference mynet.

As you can probably infer from the above, I modified your code to get the ZMQ uri from an environment variable, because this made it easier to experiment. You can find the above docker-compose.yaml and the modified code at:

• https://github.com/larsks/so-example-jimmyc

Update

In case you really want/need to have two separate docker-compose.yaml files, I've updated the example to include per-service files. These examples use the alias option to provide a name by which the client can contact the server, regardless of your local directory layout:

version: "2"

services:
  serverd:
    build: .
    command: ["python", "-u", "./serverd.py"]
    environment:
      SERVER_LISTEN_URI: tcp://0.0.0.0:5557
    networks:
      mynet:
        aliases:
          - serverd

networks:
  mynet:
    external: True

This configuration requires that you create mynet before bringing up the containers.