How do I find out why my Docker service is in Pending state?

Question

I'm playing Docker Swarm, and using a docker-compose.yml with docker deploy. All services get deployed except for one, which stays in Pending state. I have added a constraint that ties this service to one of the nodes. My question is not so much about this particular problem, but more about how to troubleshoot. The Docker documentation mentions possible causes for a service to be in pending state, but none of those apply (constraint problem, resource drainage).

Can I see the docker swarm 'thought process' somewhere? What is it thinking?

Edit: should have made it more clear that I am using the new, 1.12-introduced, swarm option.

Answer 1

I will explain how to debug when the service does not start as expected in the docker swarm mode.

First of all, get a task ID with docker service ps <service-name>.

Next, it is useful to check meta data with docker inspect <task-id>. In particular, Error message before container start is in the status field, and then reconfirm whether it was started with the intended parameters.

If the task has container ID, it was abnormally exited after starting the container, so check the log of the container with docker logs <container-id>

I hope this would be some of help.

Answer 2

docker run swarm has a --debug option which can tell you more.

See docker swarm issue 2341 or docker issue 24982 to see that option used to debug pending states.

For instance:

(unknown): 192.168.99.106:2375(node2 ip)
└ ID:
└ Status: Pending
└ Containers: 0
└ Reserved CPUs: 0 / 0
└ Reserved Memory: 0 B / 0 B
└ Labels:
└ Error: Cannot connect to the Docker daemon. Is the docker daemon running on this host?....