heterogeneous kubernetes cluster, container performance difference

Question

Suppose there is a k8s cluster which is made up of nodes of different hardware specs.

Then 2 pods are allocated to 2 different nodes.

Suppose the container in the pod had specified 1000m cpu.

Could the two containers running on those 2 different nodes show different levels of performance?

To ask more directly: in a heterogeneous k8s cluster, could containers perform differently(iops, cpu, mem, etc) due to which node it happens to be allocated to?

Answer 1

Yes. If two Pods both specify e.g. 1000m CPU and they run on different nodes, they both get an equivalent of 1 CPU core on their node. If one node has a faster CPU than the other node, then the Pod on this node correspondingly runs faster.

The same applies for other hardware characteristics of the nodes, e.g. the availability of a GPU, the speed of the memory and cache, etc.

If you want a more deterministic behaviour, you can categorise your nodes with labels or assign taints to them. Then, on the Pod side, you can influence what node a Pod gets scheduled to with tolerations, node selectors, node affinities, or even a custom scheduler.