Is it possible to run multiple nested kubernetes cluster inside kubernetes (k-in-k)

Question

Has anyone tried out nested kubernetes cluster inside kubernetes, I want to emulated sandboxed a small kubernetes cluster inside my running kubernetes cluster and not again use VM's again and again for new cluster.

New clusters I want to used for testing and trail of various features of kubernetes, has anyone tried anything on it.

I tried various solution like.

• https://github.com/marun/nkube
• https://kubernetes.io/blog/2017/01/how-we-run-kubernetes-in-kubernetes-kubeception/

I also tried https://github.com/kubernetes-sigs/kubeadm-dind-cluster and this works for me, but it starts the k8s cluster using docker containers, but I wanted this sort of solution to work inside kubernetes, so that it can be scaled across nodes I have.

If anyone has tried it or can point to solution people have tried, it can be of great help.

Edit 1:

I know k8s runs workloads as containers and this was part of my problem, "is it possible to run k8s using containers or pods or using any other k8s constructs".

I know, we can create namespace and network policies around it, but that would regulate the system, I wanted people to play with master/api-servers freely, un-regulated like we do in minikube. Wanted to have freedom to screw up master/api-servers, and to clean mess probably just need something like k8s delete cluster c1 or anything of that sort.

And for creating those clusters in sandbox cluster was just a command operation. Just didn't wanted to spin vm's again and setup cluster, if someone wants to play with it,

Answer 1

I realize this question is way old and the landscape has changed dramatically, but just in case anyone else ends up here in 2022, make sure you look into Kubernetes-in-Docker (KinD) for this kind of situation: https://kind.sigs.k8s.io/

This allows you to create local K8s clusters which run as local docker containers (e.g. local CP is a container, workers are containers, etc) and is excellent for this kind of experimental development work.

Answer 2

I understand that for some reason you don't want VM, but if you would change mind a bit, you could use https://github.com/Mirantis/virtlet/blob/master/examples/k8s.yaml which is an example how to start kubernetes in kubernetes, while nested nodes will be pods (to be more precise - VM pods, started using Virtlet as CRI compatible runtime), based on StatefulSet described in this single yaml.