kubernetes imagePullPolicy:Always is not pulling image automatically

Question

I want that every time I create a new image with the tag latest Kubernetes automatically pull the new image. I added imagePullPolicy: Always in pod spec but it doesn't update the old image with new image.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: node
  namespace: dev
  labels:
    app: my-node-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-node-app
  template:
    metadata:
      labels:
        app: my-node-app
    spec:
      hostNetwork: true
      securityContext:
        fsGroup: 1000
      containers:
      - name: node
        imagePullPolicy: Always
        image: gcr.io/my-repo/my-node-app:latest
        ports:
        - containerPort: 3000
        envFrom:
          - configMapRef:
              name: my-configmap
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 2
            memory: 8Gi
      restartPolicy: Always

Answer 1

imagePullPolicy is only taken into account by Kubernetes when a POD is created or re-started. It is NOT taken into account while a POD is running, which means it does NOT check for image updates at any time while a POD is running.

Even if another POD with the same image would be scheduled onto the same Kubernetes node, the already running POD is not affected, even though Kubernetes does a pull and then uses the new image for the new POD.

If you want the desired functionality, you will have to implement your own solution. You could do this by implementing a sidecar that regularly checks the Docker Repository for changes to the given tag. When it detects such a change, it could trigger a restart of the POD, which would then force the image to be re-pulled.

A restart of the POD can either be triggered by simply exiting the sidecar or by utilizing the Kubernetes API inside the sidecar. The latter solution however gets more complicated as you will also need service accounts and RBAC rules to get proper permissions inside the sidecar container. It also has security implications you'd have to give the whole POD escalated permissions.