Kubernetes PullImageError using Docker Hub with a private image

Question

I'm struggling to get Kubernetes to work with my private hub.docker.com registry image.

I am using kubectl version: Client Version: version.Info{Major:"1", Minor:"1+", GitVersion:"v1.1.0-alpha.0.1588+e44c8e6661c931", GitCommit:"e44c8e6661c931f7fd434911b0d3bca140e1df3a", GitTreeState:"clean"} Server Version: version.Info{Major:"1", Minor:"1", GitVersion:"v1.1.3", GitCommit:"6a81b50c7e97bbe0ade075de55ab4fa34f049dc2", GitTreeState:"clean"}

and Vagrant 1.7.4 on Mac OS X Yosemite 10.10.5

I followed the instructions given here: https://github.com/kubernetes/kubernetes/blob/release-1.1/docs/user-guide/images.md#pre-pulling-images

In a nutshell, it says you should login to the registry then base64 encode the contents of the resulting .docker/config.json, and use that in a yaml document as follows:

apiVersion: v1
kind: Secret
metadata:
  name: myregistrykey
data:
  .dockercfg: eyAiYXV0aHMiOiB7ICJodHRwczovL2luZGV4LmRvY2tlci5pby92MS8iOiB7ICJhdXRoIjogImFXNTBjbWx1YzJsak9tSTJVVTR5Z...h1YkBpbnRyaW5zaWMud29ybGQiIH0gfSB9Cg==
type: kubernetes.io/dockercfg

Then feed that to kubectl. I then used the resulting key (here called myregistrykey) in my pod definition:

apiVersion: v1
kind: Pod
metadata:
  name: authorities-backend
spec:
  containers:
    - name: authorities-backend
      image: intrinsic/authorities-backend:latest
  imagePullSecrets:
    - name: myregistrykey

and kubectl created it.

However, kubectl keeps failing to retrieve the image:

[root@kubernetes-master intrinsic]# kubectl get pods
NAME                  READY     STATUS           RESTARTS   AGE
authorities-backend   0/1       PullImageError   0          7m

docker pull on the Kubernetes master worked however.

What am I missing?

UPDATE

In the pod definition above, I had omitted to specify the registry host, i.e. docker.io. Fixing it, it becomes: image: docker.io/intrinsic/authorities-backend:latest However, the problem persists. Doing kubectl get events -w gets me: 6s 0s 2 authorities-backend Pod spec.containers{authorities-backend} Failed {kubelet 10.245.1.3} Failed to pull image "docker.io/intrinsic/authorities-backend": image pull failed for docker.io/intrinsic/authorities-backend, this may be because there are no credentials on this request. details: (Error: image intrinsic/authorities-backend:latest not found) I know the secret has been properly registered, as I have it under kubectl get secrets: NAME TYPE DATA AGE default-token-a7s5n kubernetes.io/service-account-token 2 51m myregistrykey kubernetes.io/dockercfg 1 50m

Still confused...

Candide

Answer 1

The documentation is out of date, in that it refers to .dockercfg instead of .docker/config.json. I will update it.

When you use the new .docker/config.json format, you need to set type: kubernetes.io/dockerconfigjson instead of type: kubernetes.io/.dockercfg.

Support for type: kubernetes.io/dockerconfigjson was added in v1.1.0 so it is supported by your server, but is not supported by your client (which is v1.1.0-alpha which predates v1.1.0).

When you use type: kubernetes.io/dockerconfigjson, it should validate your secret contents.

With type: kubernetes.io/dockerconfigjson, you do want to keep the auths wrapper.

Answer 2

So, I kept researching the web for an answer to my problem and eventually found this:

https://github.com/kubernetes/kubernetes/issues/7954#issuecomment-115241561

At the very end of the thread, jjw27 has nailed it. The kubernetes documentation mentions the .dockercfg.json file just to say that its contents needs to be base64-encoded. There are actually two issues with this file:

1. it looks like it morphed into another file actually, i.e. .docker/config.json
2. the auth info in this file is wrapped by an additional auths objects, which you have to get rid of.

Quoting jjw27

Did not work:

{
  "auths": {
    "hub.example.com:1024": {
      "auth": "asdf=",
      "email": "[email protected]"
     }
  }
}

Worked:

{
  "hub.example.com:1024": {
    "auth": "asdf=",
    "email": "[email protected]"
  }
}

Google, please update this doc!!

Message to Kubernetes devs #2: Also, not complaining with a malformed base64-encoded secret is very misleading. Please validate user input and complain if it contains errors.