Problem pulling images when running private docker registry inside of Kubernetes

Question

I migrated our docker registry that was running on an external dedicated server into our Kubernetes cluster

Now I can still push and pull images to the registry from every external machine but when I try to deploy images from the registry to the Kubernetes cluster itself it is not able to pull it. I get the following error log:

 Warning  Failed                 47s (x3 over 1m)  kubelet, gke-kube-1-default-pool-c5e11d0f-zxm8  Failed to pull image "myregistry.example.com/appimage:1": rpc error: code = Unknown desc = Error response from daemon: Get https://myregistry.example.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
 Warning  Failed                 47s (x3 over 1m)  kubelet, gke-kube-1-default-pool-c5e11d0f-zxm8  Error: ErrImagePull

The registry is configured to be accessible via https://myregistry.example.com by a traefik ingress controller and it looks like Kubernetes is internally trying to take a different route?

Answer 1

It's an odd path, but it should work, but taking a wild guess it looks like a DNS issue. (It works for me connecting to an externally facing service). Some things to look at:

• Can you resolve myregistry.example.com from any other running pod?
• What does the /etc/resolv.conf look like?
• What about your K8s nodes /etc/resolv.conf
• Can you resolve myregistry.example.com from your nodes?