Windows & Linux living together in Kubernetes cluster

Question

I was looking at the kubernetes documentation which seems to have a windows compability, however I don't find completely clear if both Linux and Windows can live together (I mean, in diferent VMs but the same cluster).

I would like to know if there is any support for this scenario in gcloud, azure or aws. And also, the procedure or example to make it work. Like how to create a pod in the correct VM (windows or linux) and how horizontal and cluster autoscalers work.

The use case is 2 APIs, one running in windows (.NET Framework) and other in linux (python/c++) and I want to be able to reroute them, be able to call each other, scale them and so on with kubernetes. As a note, the .NET Framework application have dependencies (mainly for mathematical optimization) that cannot be passed to .NET Core, this implies that I cannot convert the application to linux-based.

Answer 1

Some history, so containers is a Linux thing so there are no containers per se on Windows. Docker created Docker for Windows but essentially what it does is run a Hyper-V Linux VM (used to be VirtualBox) and inside it runs your containers. As of the latest Docker version, Microsoft has added capabilities on Hyper-V to allow running these containers kinda natively making it easy to run .NET apps in containers.

K8s is implemented in Golang so it was generally easier to port main components like the kubelet, kube-proxy, kubectl to Windows, by using the Golang cross-compiler (or native on Windows)

A tricky part is the networking but looks like they've got it figured out in the docs

As far as public cloud support from major providers:

• AWS

  • Hypervisor: Modified Xen or KVM. No nested virtualization support.
  • VMs: Windows VMs. Can't take advantage of Hyper-V with nested virtualization, but can run Docker for Windows.
  • Bare-metal: (i3.metal as of this writing). Run Hyper-V natively and Docker for Windows.

• Azure

  • Hypervisor: Hyper-V. Supports nested virtualization on some instance types.
  • VMs: Windows VMs, can use nested virtualization with Hyper-V, and can run Docker for Windows.
  • ACS, AKS, ACE: Should be able to take advantage of Hyper-V with nested virtualization and some cases natively.

• GCP

  • Hypervisor: KVM. Supports nested virtualization on some instance types.
  • VMs: Windows VMs. Can run Hyper-V with nested virtualization and can run Docker for Windows.

Other than that I don't know what else there's to it (Other than what's in the docs) The question is very broad. Just install Docker for Windows, setup networking, join your cluster with kubeadm and schedule your Windows workloads using the nodeSelector spec in your pods and make sure you label your Windows nodes with beta.kubernetes.io/os=windows

There's another good guide on to set up Kubernetes with Windows nodes here