docker service create never returns

Question

What would prevent docker service create ... command from completing? It never returns to the command prompt or reports any kind of error. Instead, it repeatedly switches between, "new", "assigned", "ready", and "starting".

Here's the command:

docker service create --name proxy -p 80:80 -p 443:443 -p 8080:8080 --network proxy -e MODE=swarm vfarcic/docker-flow-proxy

This is from "The DevOps 2.1 Toolkit - Docker Swarm", a book by Viktor Farcic, and I've come to a problem I can't get past...

I'm trying to run a reverse proxy service on a swarm of three nodes, all running locally on a LinuxMint system, and docker service create never returns to the command prompt. It sits there and loops between statuses and always says overall progress: 0 out of 1 tasks...

• This is on a cluster of three nodes all running locally, created with docker-machine with one manager and two workers.
• While testing, I've successfully created other overlay networks and other services using the same cluster of nodes.

As far as I can tell, the only significant differences between this and the other services I created for these nodes are:

1. There are three published ports. I double-checked the host and there's nothing else listening on any of those ports.
2. I don't have a clue what the vfarcic/docker-flow-proxy does.

I'm running this on a (real, not VirtualBox/VMWare/Hyper-V) Linux Mint 19 system (based on Ubuntu 18.04) with Docker 18.06. My host has 16Gb RAM and plenty of spare disk space. It was set up for the sole purpose of learning Docker after having too many problems on a Windows host.

Update:

If I Ctrl-C in the terminal where I tried to create the service, I get this message:

Operation continuing in background. Use docker service ps p7jfgfz8tqemrgbg1bn0d06yp to check progress.

If I use docker service ps --no-trunc p473obsbo9tgjom1fd61or5ap I get:

ID                          NAME                IMAGE                                                                                                         NODE                DESIRED STATE       CURRENT STATE               ERROR                                                          PORTS
dbzvr67uzg5fq9w64uh324dtn   proxy.1             dockerflow/docker-flow-proxy:latest@sha256:e4e4a684d703bec18385caed3b3bd0482cefdff73739ae21bd324f5422e17d93   node-1              Running             Starting 11 seconds ago
vzlttk2xdd4ynmr4lir4t38d5    \_ proxy.1         dockerflow/docker-flow-proxy:latest@sha256:e4e4a684d703bec18385caed3b3bd0482cefdff73739ae21bd324f5422e17d93   node-1              Shutdown            Failed 16 seconds ago       "task: non-zero exit (137): dockerexec: unhealthy container"
q5dlvb7xz04cwxv9hpr9w5f7l    \_ proxy.1         dockerflow/docker-flow-proxy:latest@sha256:e4e4a684d703bec18385caed3b3bd0482cefdff73739ae21bd324f5422e17d93   node-1              Shutdown            Failed 49 seconds ago       "task: non-zero exit (137): dockerexec: unhealthy container"
z86h0pcj21joryb3sj59mx2pq    \_ proxy.1         dockerflow/docker-flow-proxy:latest@sha256:e4e4a684d703bec18385caed3b3bd0482cefdff73739ae21bd324f5422e17d93   node-1              Shutdown            Failed about a minute ago   "task: non-zero exit (137): dockerexec: unhealthy container"
uj2axgfm1xxpdxrkp308m28ys    \_ proxy.1         dockerflow/docker-flow-proxy:latest@sha256:e4e4a684d703bec18385caed3b3bd0482cefdff73739ae21bd324f5422e17d93   node-1              Shutdown            Failed about a minute ago   "task: non-zero exit (137): dockerexec: unhealthy container"

So, no mention of node-2 or node-3 and I don't know what the error message means.

If I then use docker service logs -f proxy I get the following messages repeating every minute or so:

proxy.1.l86x3a6md766@node-1 | 2018/09/11 12:48:46 Starting HAProxy proxy.1.l86x3a6md766@node-1 | 2018/09/11 12:48:46 Getting certs from http://202.71.99.195:8080/v1/docker-flow-proxy/certs

I can only guess that it fails to get whatever certs it's looking for. There doesn't seem to be anything at that address, but an IP address lookup shows that it leads to my ISP. Where's that coming from?

Answer 1

There may be more than one motives why docker can't create the service. The simplest to see why is to see the last status of the service by running:

docker service ps --no-trunc proxy

where you have an ERROR column that describes the last error.

Alternatively you can run

docker service logs -f proxy

to see what's going on inside the proxy service.