How to restart kubernetes nodes?

Question

The status of nodes is reported as unknown

"conditions": [           {             "type": "Ready",             "status": "Unknown",             "lastHeartbeatTime": "2015-11-12T06:03:19Z",             "lastTransitionTime": "2015-11-12T06:04:03Z",             "reason": "Kubelet stopped posting node status."           } 

whle kubectl get nodes return a NOTReady status. What does this imply and how to fix this?

Answer 1

Get nodes

kubectl get nodes 

Result:

NAME            STATUS     AGE 192.168.1.157   NotReady   42d 192.168.1.158   Ready      42d 192.168.1.159   Ready      42d 

Describe node

Here is a NotReady on the node of 192.168.1.157. Then debugging this notready node, and you can read offical documents - Application Introspection and Debugging.

kubectl describe node 192.168.1.157 

Partial Result:

Conditions: Type          Status          LastHeartbeatTime                       LastTransitionTime                      Reason                  Message ----          ------          -----------------                       ------------------                      ------                  ------- OutOfDisk     Unknown         Sat, 28 Dec 2016 12:56:01 +0000         Sat, 28 Dec 2016 12:56:41 +0000         NodeStatusUnknown       Kubelet stopped posting node status. Ready         Unknown         Sat, 28 Dec 2016 12:56:01 +0000         Sat, 28 Dec 2016 12:56:41 +0000         NodeStatusUnknown       Kubelet stopped posting node status. 

There is a OutOfDisk on my node, then Kubelet stopped posting node status. So, I must free some disk space, using the command of df on my Ubuntu14.04 I can check the details of memory, and using the command of docker rmi image_id/image_name under the role of su I can remove the useless images.

Login in node

Login in 192.168.1.157 by using ssh, like ssh [email protected], and switch to the 'su' by sudo su;

Restart kubelet

/etc/init.d/kubelet restart 

Result:

stop: Unknown instance:  kubelet start/running, process 59261 

Get nodes again

On the master:

kubectl get nodes 

Result:

NAME            STATUS    AGE 192.168.1.157   Ready     42d 192.168.1.158   Ready     42d 192.168.1.159   Ready     42d 

Ok, that node works fine.

Here is a reference: Kubernetes