Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kubernetes Job Cleanup

From what I understand the Job object is supposed to reap pods after a certain amount of time. But on my GKE cluster (Kubernetes 1.1.8) it seems that "kubectl get pods -a" can list pods from days ago.

All were created using the Jobs API.

I did notice that after delete the job with kubectl delete jobs The pods were deleted too.

My main concern here is that I am going to run thousands and tens of thousands of pods on the cluster in batch jobs, and don't want to overload the internal backlog system.

like image 972
Lior Regev Avatar asked Apr 03 '16 11:04

Lior Regev


People also ask

How do I get a job cleaning Kubernetes?

Delete the job with kubectl (e.g. kubectl delete jobs/pi or kubectl delete -f ./job. yaml ). When you delete the job using kubectl , all the pods it created are deleted too.

How do I remove a Cronjob from Kubernetes?

You can delete them at once with kubectl delete jobs --all , if you want to delete all jobs in the current namespace (not just the ones created by "hello".)

What is TTL in Kubernetes?

FEATURE STATE: Kubernetes v1.23 [stable] TTL-after-finished controller provides a TTL (time to live) mechanism to limit the lifetime of resource objects that have finished execution.

How do you clean up Kubernetes pods?

If you create pods directly (not via a deployment), you can delete them directly, and they will stay deleted. Pods (that were created directly), deployments, and services can all be deleted independently of one another, order doesn't matter. If you want to delete them but not the namespace, delete them in any order.


1 Answers

It looks like starting with Kubernetes 1.6 (and the v2alpha1 api version), if you're using cronjobs to create the jobs (that, in turn, create your pods), you'll be able to limit how many old jobs are kept. Just add the following to your job spec:

successfulJobsHistoryLimit: X failedJobsHistoryLimit: Y 

Where X and Y are the limits of how many previously run jobs the system should keep around (it keeps jobs around indefinitely by default [at least on version 1.5.])

Edit 2018-09-29:

For newer K8S versions, updated links with documentation for this are here:

  • CronJob - Job History Limits

  • CronJob API Spec

like image 178
JJC Avatar answered Oct 11 '22 11:10

JJC