GKE extended memory and Kubernetes memory allocatable

Question

When creating a cluster on GKE its possible to create Custom Instance Types. When adding 8GB of memory to an n1-standard-1 Kubernetes only shows memory allocatable of 6.37GB. Why is this?

The requested memory includes all the pods in kube-system namespace so where is this extra memory going?

Answer 1

Quotinig from documentation:

Node allocatable resources

Note that some of a node's resources are required to run the Kubernetes Engine and Kubernetes resources necessary to make that node function as part of your cluster. As such, you may notice a disparity between your node's total resources (as specified in the machine type documentation) and the node's allocatable resources in Kubernetes Engine

Note: As larger machine types tend to run more containers (and by extension, Kubernetes pods), the amount of resources that Kubernetes Engine reserves for cluster processes scales upward for larger machines.

Caution: In Kubernetes Engine node versions prior to 1.7.6, reserved resources were not counted against a node's total allocatable resources. If your nodes have recently upgraded to version 1.7.6, they might appear to have fewer resources available, as Kubernetes Engine now displays allocatable resources. This can potentially lead to your cluster's nodes appearing overcommitted, and you might want to resize your cluster as a result.

For example performing some tests you can doublecheck:

Machine type            Memory(GB)  Allocatable(GB) CPU(cores)  Allocatable(cores)

g1-small                     1.7        1.2          0.5         0.47
n1-standard-1 (default)      3.75       2.7          1           0.94
n1-standard-2                7.5        5.7          2           1.93
n1-standard-4               15         12            4           3.92
n1-standard-8               30         26.6          8           7.91
n1-standard-16              60         54.7         16          15.89

Note: The values listed for allocatable resources do not account for the resources used by kube-system pods, the amount of which varies with each Kubernetes release. These system pods generally occupy an additional 400m CPU and 400mi memory on each node (values are approximate). It is recommended that you directly inspect your cluster if you require an exact accounting of usable resources on each node.

UPDATE

There is also the official explanation from the Kubernetes Documentation regarding why this resources are used:

kube-reserved is meant to capture resource reservation for kubernetes system daemons like the kubelet, container runtime, node problem detector, etc. It is not meant to reserve resources for system daemons that are run as pods. kube-reserved is typically a function of pod density on the nodes. This performance dashboard exposes cpu and memory usage profiles of kubelet and docker engine at multiple levels of pod density. This blog post explains how the dashboard can be interpreted to come up with a suitable kube-reserved reservation.

I would suggest you to go thorugh this page if you are interested to learn more.