Kubernetes: Failed to pull image. Server gave HTTP response to HTTPS client

Question

I'm trying to use Kubernetes with Docker. My image runs with Docker. I have one master-node and two worker-nodes. I also created a local registry like this $ docker run -d -p 5000:5000 --restart=always --name registry registry:2 and pushed my image into it. Everything worked fine so far.

I added { "insecure-registries":["xxx.xxx.xxx.xxx:5000"] } to the daemon.json file at /etc/docker. And I also changed the content of the docker-file at /etc/default/to DOCKER_OPTS="--config-file=/etc/docker/daemon.json". I made the changes on all nodes and I restarted the docker daemon afterwards.

I am able to pull my image from every node with the following command:

sudo docker pull xxx.xxx.xxx.xxx:5000/helloworldimage

I try to create my container from the master node with the command bellow:

sudo kubectl run test --image xxx.xxx.xxx.xxx:5000/helloworldimage

Than I get the following error:

Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  <unknown>          default-scheduler  Successfully assigned default/test-775f99f57-m9r4b to rpi-2
  Normal   BackOff    18s (x2 over 44s)  kubelet, rpi-2     Back-off pulling image "xxx.xxx.xxx.xxx:5000/helloworldimage"
  Warning  Failed     18s (x2 over 44s)  kubelet, rpi-2     Error: ImagePullBackOff
  Normal   Pulling    3s (x3 over 45s)   kubelet, rpi-2     Pulling image "xxx.xxx.xxx.xxx:5000/helloworldimage"
  Warning  Failed     3s (x3 over 45s)   kubelet, rpi-2     Failed to pull image "xxx.xxx.xxx.xxx:5000/helloworldimage": rpc error: code = Unknown desc = failed to pull and unpack image "xxx.xxx.xxx.xxx:5000/helloworldimage:latest": failed to resolve reference "xxx.xxx.xxx.xxx:5000/helloworldimage:latest": failed to do request: Head https://xxx.xxx.xxx.xxx:5000/v2/helloworldimage/manifests/latest: http: server gave HTTP response to HTTPS client
  Warning  Failed     3s (x3 over 45s)   kubelet, rpi-2     Error: ErrImagePull

This is the docker version I use:

Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:37:22 2019
 OS/Arch:           linux/arm
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea
  Built:            Wed Nov 13 07:31:17 2019
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

This is the Kubernetes version I use:

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0+k3s.1", GitCommit:"0f644650f5d8e9f091629f860b342f221c46f6d7", GitTreeState:"clean", BuildDate:"2020-01-06T23:20:30Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/arm"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0+k3s.1", GitCommit:"0f644650f5d8e9f091629f860b342f221c46f6d7", GitTreeState:"clean", BuildDate:"2020-01-06T23:20:30Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/arm"}```

Answer 1

Kubernetes: Failed to pull image. Server gave HTTP response to HTTPS client.

{ "insecure-registries":["xxx.xxx.xxx.xxx:5000"] }

to the daemon.json file at /etc/docker.

I solved this problem by configuring it on all kubernetes nodes.