Install two traefik ingress controller on same kubernetes Cluster

Question

I have a situation, where I am planning to use two separate traefik ingress controller inside the Kubernetes cluster.

I have a few URLs which I want to be accessible through VPN only, and few which can be publicly accessible.

In the current architecture, I have one traefik-ingress controller, and two separate ALBs, one internal and one internet-facing, both pointing to traefik.
Let's say, I have a URL public.example.com and private.example.com. public.example.com is pointing to internet-facing ALB, and private.example.com is pointing to internal ALB. But what if someone get to know the pointing of public.example.com and points private.example.com to same pointing in his /etc/hosts, he will be able to access my private website.

To avoid this, I am planning to run two separate traefik-ingress-controller, one which will be serving only private URL and one public URL. Can this be done? Or is there any other way to avoid this

Answer 1

To deploy two separate traefik-ingress controller, to serve private and public traffic separately, I used kubernetes.ingressclass=traefik args.

This is what documentation has to say for kubernetes.ingressclass:

--kubernetes.ingressclass  Value of kubernetes.io/ingress.class annotation to watch for

I created two deployment, having separate value for kubernetes.ingressclass.

One with kubernetes.ingressclass=traefik, which was behind a public ALB and kubernetes.ingressclass=traefik-internal, which was behind a private/internal ALB

For services, which I want to serve privately, I use the following annotations in ingress objects :

annotations:
    kubernetes.io/ingress.class: traefik-internal

and for public

annotations:
  kubernetes.io/ingress.class: traefik

My deployment.yaml

---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik-internal-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-internal-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-internal-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-internal-ingress-lb
    spec:
      serviceAccountName: traefik-internal-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:v1.7
        name: traefik-internal-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
        - --accesslog=true
        - --kubernetes.ingressclass=traefik-internal ##this makes it to watch only for ingress objects with annotaion "kubernetes.io/ingress.class: traefik-internal"

Hope this helps someone.

Answer 2

You can achieve this with a single Ingress controller inside the cluster but by creating various Ingress Kubernetes Objects.

For Private site:- consider whitelist-source-range annotation in the ingress resource.

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#whitelist-source-range

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 10.0.0.0/24,172.10.0.1
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          serviceName: test
          servicePort: 80

For Public site:-

https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          serviceName: test
          servicePort: 80