(Kubernetes + Minikube) can't get docker image from local registry

Question

I have setup docker on my machine and also minikube which have docker inside it, so probably i have two docker instances running on different VM

I build an image and tag it then push it to local registry and it pushed successfully and i can pull it from registry too and also when i run curl to get tags list i got result, and here are what i did

1- docker build -t 127.0.0.1:5000/eliza/console:0.0.1 .
2- docker run -d -p 5000:5000 --name registry registry:2
3- docker tag a3703d02a199 127.0.0.1:5000/eliza/console:0.0.1
4- docker push 127.0.0.1:5000/eliza/console:0.0.1
5- curl -X GET http://127.0.0.1:5000/v2/eliza/console/tags/list

all above steps are working fine with no problems at all.

My problem is when i run minikube and try to access this image in local registry inside it

So when i run next commands

1- sudo minikube start --insecure-registry 127.0.0.1:5000
2- eval $(minikube docker-env)
3- minikube ssh
4- curl -X GET http://127.0.0.1:5000/v2/eliza/console/tags/list

in last step (point 4) it gave me next message

curl: (7) Failed to connect to 127.0.0.1 port 5000: Connection refused

So i can access image registry from my machine but not from minikube which make a problems of course with me when i deploy this image using Kubernetes on minikube and make deploy failed due to can't connect to http://127.0.0.1:5000

Can you help me configuring minikube to see my local registry so my problem will be solved then i can deploy image to minikube using kubernetes successfully?

UPDATE

I am using this yaml file (i named it ConsolePre.yaml) to deploy my image using kubernetes

apiVersion: v1
  kind: Service
  metadata:
    name: tripbru-console
    labels:
      app: tripbru-console
  spec:
    ports:
      - port: 9080
        targetPort: 9080
        nodePort: 30181
    selector:
      app: tripbru-console
      tier: frontend
    type: NodePort
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: tripbru-console
  labels:
    app: tripbru-console
spec:
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: tripbru-console
        tier: frontend
    spec:
      containers:
      - image: docker.local:5000/eliza/console:0.0.1
        name: tripbru-console
        ports:
        - containerPort: 9080
          name: tripbru-console

and when i run next command to apply changes

sudo kubectl apply -f /PATH_TO_YAML_FILE/ConsolePre.yaml

the result is

NAME                                      READY     STATUS         RESTARTS   AGE
po/tripbru-console-1655054400-x3g87       0/1       ErrImagePull   0          1m

and when i run describe command

sudo kubectl describe pod tripbru-console-1655054400-x3g87

i found next message in description result

Error response from daemon: {"message":"Get https://docker.local:5000/v1/_ping: dial tcp: lookup docker.local on 10.0.2.3:53: read udp 10.0.2.15:57792-\u003e10.0.2.3:53: i/o timeout"}

and i configured docker.local xxx.xxx.xx.4 in minikube /etc/hosts so i don't know from where 10.0.2.3:53 and 10.0.2.15:57792 come from.

So how can i solve this issue too.

Thanks :)

Answer 1

The issue is your notion using 127.0.0.1 anywhere you want. This is wrong.

So if your machine IP is 192.168.0.101. Then below works

1- docker build -t 127.0.0.1:5000/eliza/console:0.0.1 .
2- docker run -d -p 5000:5000 --name registry registry:2
3- docker tag a3703d02a199 127.0.0.1:5000/eliza/console:0.0.1
4- docker push 127.0.0.1:5000/eliza/console:0.0.1
5- curl -X GET http://127.0.0.1:5000/v2/eliza/console/tags/list

Because docker run maps the registry to 127.0.0.1:5000 and 192.168.0.101:5000. Now on your machine only this 127.0.0.1 will work. Now when you use

3- minikube ssh

You get inside the minikube machine and that doesn't have a registry running on 127.0.0.1:5000. So the error. The registry is no reachable inside this machine using the machine IP.

The way I usually solve this is issue is by using host name both locally and inside the other VMs.

So on your machine create a entry in /etc/hosts

docker.local 127.0.0.1

And change your commands to

1- docker build -t docker.local:5000/eliza/console:0.0.1 .
2- docker run -d -p 5000:5000 --name registry registry:2
3- docker tag a3703d02a199 docker.local:5000/eliza/console:0.0.1
4- docker push docker.local:5000/eliza/console:0.0.1
5- curl -X GET http://docker.local:5000/v2/eliza/console/tags/list

And then when you use minikube ssh, make a entry for docker.local in /etc/hosts

docker.local 192.168.0.101

Then curl -X GET http://docker.local:5000/v2/eliza/console/tags/list

Edit-1

For the TLS issue you need to Stop the docker service inside minikube

systemctl stop docker

Then edit /etc/systemd/system/docker.service.d/10-machine.conf and change

ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox --insecure-registry 10.0.0.0/24

to

ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox --insecure-registry 10.0.0.0/24 --insecure-registry docker.local:5000 --insecure-registry 192.168.1.4:5000

Then reload daemon and start the docker service

systemctl daemon-reload
systemctl start docker

After that try to pull

docker pull docker.local:5000/eliza/console:0.0.1

And the command should work

Answer 2

How to access Processes running on hostmachine from with in a Docker container?

It is a popular question in the docker-land. See here. https://stackoverflow.com/a/24326540/6785908 There are other ways too, for example, For Docker on mac, docker.for.mac.localhost DNS name will resolve to the hostmachine

From https://docs.docker.com/docker-for-mac/networking/#i-cannot-ping-my-containers

The Mac has a changing IP address (or none if you have no network access). From 17.06 onwards our recommendation is to connect to the special Mac-only DNS name docker.for.mac.localhost which will resolve to the internal IP address used by the host.

Assuming that primary purpose of this minikube is for local testing, there is an easier way deploy your docker container (This doesnt even need a local docker registry)

Method 2: Point your docker CLI to Docker daemon running within your minikube and then execute docker build command there.

First thing to understand here is, when you install docker in your machine, it has 2 parts, 1) a docker cli with which you can interact with docker daemon 2) A docker daemon. In this method we will point our local docker cli to minikube's docker daemon and execute docker build.

https://github.com/kubernetes/kubernetes.github.io/blob/master/docs/getting-started-guides/minikube.md#reusing-the-docker-daemon

quoting relevant parts here

When using a single VM of Kubernetes, it's really handy to reuse the minikube's built-in Docker daemon; as this means you don't have to build a docker registry on your host machine and push the image into it - you can just build inside the same docker daemon as minikube which speeds up local experiments. Just make sure you tag your Docker image with something other than 'latest' and use that tag while you pull the image. Otherwise, if you do not specify version of your image, it will be assumed as :latest, with pull image policy of Always correspondingly, which may eventually result in ErrImagePull as you may not have any versions of your Docker image out there in the default docker registry (usually DockerHub) yet.

To be able to work with the docker daemon on your mac/linux host use the docker-env command in your shell:

eval $(minikube docker-env)

You should now be able to use docker on the command line on your host mac/linux machine talking to the docker daemon inside the minikube VM:

do a docker container list command : docker ps. It should display even the containers related to kubernetes system (because now your cli is pointed to a docker daemon where your minikube is running).

Now build your docker image. Then it will be available in the minikube for you.