Kubernetes cpu requests/limits in heterogeneous cluster

Question

Kubernetes allows to specify the cpu limit and/or request for a POD.

Limits and requests for CPU resources are measured in cpu units. One cpu, in Kubernetes, is equivalent to:

1 AWS vCPU
1 GCP Core
1 Azure vCore
1 IBM vCPU
1 Hyperthread on a bare-metal Intel processor with Hyperthreading

Unfortunately, when using an heterogeneous cluster (for instance with different processors), the cpu limit/request depends on the node on which the POD is assigned; especially for real time applications.

If we assume that:

• we can compute a fined tuned cpu limit for the POD for each kind of hardware of the cluster
• we want to let the Kubernetes scheduler choose a matching node in the whole cluster

Is there a way to launch the POD so that the cpu limit/request depends on the node chosen by the Kubernetes scheduler (or a Node label)?

The obtained behavior should be (or equivalent to):

• Before assigning the POD, the scheduler chooses the node by checking different cpu requests depending on the Node (or a Node Label)
• At runtime, Kublet checks a specific cpu limit depending on the Node (or a Node Label)

Answer 1

No, you can't have different requests per node type. What you can do is create a pod manifest with a node affinity for a specific kind of node, and requests which makes sense for that node type. For a second kind of node, you will need a second pod manifest which makes sense for that node type. These pod manifests will differ only in their affinity spec and resource requests - so it would be handy to parameterize them. You could do this with Helm, or write a simple script to do it.

This approach would let you launch a pod within a subset of your nodes with resource requests which make sense on those nodes, but there's no way to globally adjust its requests/limits based on where it ends up.