Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Graceful termination of kubernetes pods

Tags:

We have an application with 4 pods running with a load balancer! We want to try the rolling update, but we are not sure what happens when a pod goes down! The documentation is unclear! Particularly this quote from Termination Of Pods:

Pod is removed from endpoints list for service, and are no longer considered part of the set of running pods for replication controllers. Pods that shutdown slowly can continue to serve traffic as load balancers (like the service proxy) remove them from their rotations.

So, if someone can guide us on the following questions :

1.) When a pod is shutting down, can it still serve new requests? Or does the load balancer not consider it?

2.) Does it complete the requests it is processing till the grace-period is exhausted? and then kills the container even if any process is still running?

3.) Also, this mentions replication controllers, what we have is a Deployment and Deployment has replica sets, so will there be any difference?

We went through this question but the answers are conflicting without any source : Does a Kubernetes rolling-update gracefully remove pods from a service load balancer

like image 760
Nagireddy Hanisha Avatar asked Feb 20 '17 04:02

Nagireddy Hanisha


People also ask

How do you gracefully terminate a pod in Kubernetes?

Forced Pod termination By default, all deletes are graceful within 30 seconds. The kubectl delete command supports the --grace-period=<seconds> option which allows you to override the default and specify your own value. Setting the grace period to 0 forcibly and immediately deletes the Pod from the API server.

What is graceful shutdown in Kubernetes?

Graceful Node Shutdown allows Kubernetes to detect when a node is shutting down cleanly, and handles that situation appropriately. A Node Shutdown can be "graceful" only if the node shutdown action can be detected by the kubelet ahead of the actual shutdown.

How do I terminate a container in Kubernetes?

You send a command or API call to terminate the Pod. Kubernetes updates the Pod status to reflect the time after which the Pod is to be considered "dead" (the time of the termination request plus the grace period). Kubernetes marks the Pod state as "Terminating" and stops sending traffic to the Pod.


1 Answers

1) when a Pod is shutting down it's state is changed to Terminating and it is not considered by the LoadBalancer - as described in the Pod termination docs

2) Yes - you might want to look at the pod.Spec.TerminationGracePeriodSeconds configuration to gain some control. You'll find details in the API documentation

3) No - the ReplicaSet and the Deployment take care of scheduling Pods, there's no difference when it comes to the shutdown behaviour of the Pods

like image 60
pagid Avatar answered Sep 22 '22 10:09

pagid