Menu
i saw that some people use Kubernetes co-exist with slurm, I was just curious as to why you need kubernetes with slurm? what is the main difference between kubernetes and slurm?
895
Slurm provides resource management for the processors allocated to a job, so that multiple job steps can be simultaneously submitted and queued until there are available resources within the job's allocation.
The main difference between an HPC workload and the type of application for which Kubernetes was built is that, while HPC workloads run to complete a complex task in the shortest possible time (even if this is a long time), Kubernetes is optimized for continuously running applications.
In general, Kubernetes cluster can support both the Singularity runtime and the Docker runtime.
Kubernetes defines a network model called the container network interface (CNI), but the actual implementation relies on network plugins. The network plugin is responsible for allocating internet protocol (IP) addresses to pods and enabling pods to communicate with each other within the Kubernetes cluster.
Slurm is open source job scheduling system for large and small Linux clusters. It is mainly used as Workload Manager/Job scheduler. Mostly used in HPC (High Performance Computing) and sometimes in BigData.
Kubernetes is an orchestration system for Docker containers using the concepts of ”labels” and ”pods” to group containers into logical units. It was mainly created to run micro-services and AFAIK currently Kubernetes is not supporting Slurm.
Slumr as Job scheduler have more scheduling options than Kubernetes, but K8s is container orchestration system not only Job scheduler. For example Kubernetes is supporting Array jobs and Slurm supports Parallel and array jobs. If you want to dive in to scheduling check this article.
As I mentioned before, Kubernetes is more focused on container orchestration and Slumr is focused on Job/Workload scheduling. Only thing comes to my mind is that someone needed very personal-customized cluster using WLM-Operator + K8s + Slurm + Singularity to execute HPC/BigData jobs.
Usually Slurm Workload Manager is used by many of the world's supercomputers to optimize locality of task assignments on parallel computers.
120
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
