Kubernetes - Where does it store secrets and how does it use those secrets on multiple nodes?

Question

Not really a programming question but quite curious to know how does Kubernetes or Minikube manage secrets & uses it on multiple nodes/pods?

Let's say if I create a secret to pull image with kubectl as below -

$ kubectl create secret docker-registry regsecret --docker-server=https://index.docker.io/v1/ --docker-username=$USERNM --docker-password=$PASSWD [email protected]

What processes will occur in the backend and how will k8s or Minikube use those on multiple nodes/pods?

Answer 1

All data in Kubernetes is managed by the API Server component that performs CRUD operations on the data store (current only option is etcd).

When you submit a secret with kubectl to the API Server it stores the resource and data in etcd. It is recommended to enable encryption for secrets in in the API Server (through setting the right flags) so that the data is encrypted at rest, otherwise anyone with access to etcd will be able to read your secrets in plain text.

When the secret is needed for either mounting in a Pod or in your example for pulling a Docker image from a private registry, it is requested from the API Server by the node-local kubelet and kept in tmpfs so it never touches any hard disk unencrypted.

Here another security recommendation comes into play, which is called Node Authorization (again set up by setting the right flags and distributing certificates to API Server and Kubelets). With Node Authorization enabled you can make sure that a kubelet can only request resources (incl. secrets) that are meant to be run on that specific node, so a hacked node just exposes the resources on that single node and not everything.