Docker: Swarm worker nodes not finding locally built image

Question

Maybe I missed something, but I made a local docker image. I have a 3 node swarm up and running. Two workers and one manager. I use labels as a constraint. When I launch a service to one of the workers via the constraint it works perfectly if that image is public.

That is, if I do:

docker service create --name redis --network my-network  --constraint node.labels.myconstraint==true redis:3.0.7-alpine

Then the redis service is sent to one of the worker nodes and is fully functional. Likewise, if I run my locally built image WITHOUT the constraint, since my manager is also a worker, it gets scheduled to the manager and runs perfectly well. However, when I add the constraint it fails on the worker node, from docker service ps 2l30ib72y65h I see:

... Shutdown       Rejected 14 seconds ago  "No such image: my-customized-image"

Is there a way to make the workers have access to the local images on the manager node of the swarm? Does it use a specific port that might not be open? If not, what am I supposed to do - run a local repository?

Answer 1

The manager node doesn't share out the local images from itself. You need to spin up a registry server (or user hub.docker.com). The effort needed for that isn't very significant:

# first create a user, updating $user for your environment:
if [ ! -d "auth" ]; then
  mkdir -p auth
fi
touch auth/htpasswd
chmod 666 auth/htpasswd
docker run --rm -it \
  -v `pwd`/auth:/auth \
  --entrypoint htpasswd registry:2 -B /auth/htpasswd $user
chmod 444 auth/htpasswd

# then spin up the registry service listening on port 5000
docker run -d -p 5000:5000 --restart=always --name registry \
  -v `pwd`/auth/htpasswd:/auth/htpasswd:ro \
  -v `pwd`/registry:/var/lib/registry \
  -e "REGISTRY_AUTH=htpasswd" \
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Local Registry" \
  -e "REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd" \
  -e "REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry" \
  registry:2

# then push your image
docker login localhost:5000
docker tag my-customized-image localhost:5000/my-customized-image
docker push localhost:5000/my-customized-image

# then spin up the service with the new image name
# replace registryhost with ip/hostname of your registry Docker host
docker service create --name custom --network my-network \
  --constraint node.labels.myconstraint==true --with-registry-auth \
  registryhost:5000/my-customized-image

Answer 2

For me, this step-by-step guide worked. However, it is insecure:

# Start your registry
$ docker run -d -p 5000:5000 --name registry registry:2

# Tag the image so that it points to your registry
$ docker tag my_existing_image localhost:5000/myfirstimage

# Push it to local registry/repo
$ docker push localhost:5000/myfirstimage

# For verification you can use this command:
$ curl -X GET http://localhost:5000/v2/_catalog
# It will print out all images on repo.

# On private registry machine add additional parameters to enable insecure repo:
ExecStart=/usr/bin/dockerd --insecure-registry IP_OF_CURRENT_MACHINE:5000

# Flush changes and restart Docker:
$ systemctl daemon-reload
$ systemctl restart docker.service

# On client machine we should say docker that this private repo is insecure, so create or modifile the file '/etc/docker/daemon.json':
{ "insecure-registries":["hostname:5000"] }

# Restart docker:
$ systemctl restart docker.service

# On swarm mode, you need to point to that registry, so use host name instead, for example: hostname:5000/myfirstimage