Kubernetes - identical jobs, different parameters

Question

What's the easiest way to run a configurable number of identical jobs on Kubernetes but give each of them different parameters (like job number)?

Answer 1

1) You could either just have a template job and use bash expansions to have multiple job specifications based of that initial template.

As shown in the official Parallel Processing using Expansions user guide:

mkdir ./jobs
for i in apple banana cherry
do
  cat job.yaml.txt | sed "s/\$ITEM/$i/" > ./jobs/job-$i.yaml
done
kubectl create -f ./jobs

---

2) Or you could create a queue and have a specified number of parallel workers/jobs to empty the queue. The contents of the queue would then be the input for each worker and Kubernetes could spawn parallel jobs. That's best described in the Coarse Parallel Processing using a Work Queue user guide.

---

• The first approach is simple and straight forward but lacks flexibility
• The second requires a message queue as "overhead" but you'll gain flexibility