Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kubernetes scale down specific pods

I have a set of Pods running commands that can take up to a couple seconds. There is a process that keeps track of open request & which Pod the request is running on. I'd like the use that information when scaling down pods - either by specifying which pods to try to leave up, or specifying which pods to shut down. Is it possible to specify this type of information when changing the # of replicas, e.g. I want X replicas, try not to kill my long running tasks on pods A, B, C?

like image 964
Charles L. Avatar asked Nov 09 '15 19:11

Charles L.


People also ask

How do you scale up and down pods in Kubernetes?

You can autoscale Deployments based on CPU utilization of Pods using kubectl autoscale or from the GKE Workloads menu in the Google Cloud console. kubectl autoscale creates a HorizontalPodAutoscaler (or HPA) object that targets a specified resource (called the scale target) and scales it as needed.

How do you scale pods in Kubernetes command?

The kubectl scale command is used to change the number of running replicas inside Kubernetes deployment, replica set, replication controller, and stateful set objects. When you increase the replica count, Kubernetes will start new pods to scale up your service.

Can we scale pods in Kubernetes?

Scaling overviewKubernetes also supports autoscaling of Pods, but it is outside of the scope of this tutorial. Scaling to zero is also possible, and it will terminate all Pods of the specified Deployment. Running multiple instances of an application will require a way to distribute the traffic to all of them.


3 Answers

You can annotation specific pod with controller.kubernetes.io/pod-deletion-cost: -999 and enable PodDeletionCost featuregate. This feature is implement alpha in 1.21 and beta in 1.22.

controller.kubernetes.io/pod-deletion-cost annotation can be set to offer a hint on the cost of deleting a pod compared to other pods belonging to the same ReplicaSet. Pods with lower deletion cost are deleted first.

https://github.com/kubernetes/kubernetes/pull/99163 https://github.com/kubernetes/kubernetes/pull/101080

like image 96
wow qing Avatar answered Nov 18 '22 04:11

wow qing


This isn't currently possible. When you scale down the number of replicas, the system will choose one to remove; there isn't a way to "hint" at which one you'd like it to remove.

One thing you can do is you can change the labels on running pods which can affect their membership in the replication controller. This can be used to quarantine pods that you want to debug (so that they won't be part of a service or removed by a scaling event) but might also be possible to use for your use case.

like image 40
Robert Bailey Avatar answered Nov 18 '22 02:11

Robert Bailey


i've been looking for a solution to this myself, and i also can't find one out of the box. however, there might be a workaround (would love it if you could test and confirm)

steps:
1. delete replication controller
2. delete X desired pods
3. recreate replication controller of size X

like image 40
nathan g Avatar answered Nov 18 '22 04:11

nathan g