I have searched some but could not get much as I am new to k8s. My pods are evicted and I get message as:
"Status: Failed
Reason: Evicted
Message: The node was low on resource: nodefs."
Any help how can I figure out what is going on?
Run kubectl describe pod <pod name> and look for the node name of this pod. Followed by kubectl describe node <node-name> that will show what type of resource cap the node is hitting under Conditions: section. From my experience this happens when the host node runs out of disk space.
The command result reveals that many pods have evicted status after running kubectl get pods. The results of the check will be saved in the node's kubelet logs. To find the information, run cat /var/paas/sys/log/kubernetes/kubelet. log | grep -i Evicted -C3.
This one will start to failed Pods until the node's used resources are under the eviction threshold, which means that the Kubelet will terminate all Pod's containers and set its PodPhase as Failed. If a Deployment manages the evicted Pod, the Deployment creates another Pod to be scheduled by Kubernetes.
Run kubectl describe pod <pod name>
and look for the node name of this pod. Followed by kubectl describe node <node-name>
that will show what type of resource cap the node is hitting under Conditions:
section.
From my experience this happens when the host node runs out of disk space.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With