Getting `Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io` error with Traefikv2.3

Question

I'm trying to use Traefik Kubernetes Ingress. I'm using traefik:v2.3. K8's cli version is v1.18.3 and server version is v1.18.6IKS. I'm using IBM Kubernetes services to deploy this. But I'm getting below errors in pod logs. I'm following offical link

complete Logs

time="2020-07-26T17:01:04Z" level=info msg="Configuration loaded from flags."
time="2020-07-26T17:01:04Z" level=info msg="Traefik version 2.3.0-rc2 built on 2020-07-15T20:22:27Z"
time="2020-07-26T17:01:04Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{},\"http\":{\"redirections\":{\"entryPoint\":{\"to\":\"websecure\",\"scheme\":\"https\",\"permanent\":true,\"priority\":2147483647}}}},\"websecure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{},\"http\":{}}},\"providers\":{\"providersThrottleDuration\":2000000000,\"kubernetesIngress\":{}},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"accessLog\":{\"format\":\"common\",\"filters\":{},\"fields\":{\"defaultMode\":\"keep\",\"headers\":{\"defaultMode\":\"drop\"}}},\"certificatesResolvers\":{\"letsencrypt\":{\"acme\":{\"email\":\"[email protected]\",\"caServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"storage\":\"/data/acme.json\",\"keyType\":\"RSA4096\",\"tlsChallenge\":{}}}}}"
time="2020-07-26T17:01:04Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/contributing/data-collection/\n"
time="2020-07-26T17:01:04Z" level=info msg="Starting provider *ingress.Provider {}"
time="2020-07-26T17:01:04Z" level=debug msg="Using Ingress label selector: \"\"" providerName=kubernetes
time="2020-07-26T17:01:04Z" level=info msg="ingress label selector is: \"\"" providerName=kubernetes
time="2020-07-26T17:01:04Z" level=info msg="Creating in-cluster Provider client" providerName=kubernetes
time="2020-07-26T17:01:04Z" level=info msg="Starting provider *traefik.Provider {}"
time="2020-07-26T17:01:04Z" level=debug msg="Configuration received from provider internal: {\"http\":{\"routers\":{\"web-to-websecure\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"redirect-web-to-websecure\"],\"service\":\"noop@internal\",\"rule\":\"HostRegexp(`{host:.+}`)\",\"priority\":2147483647}},\"services\":{\"noop\":{}},\"middlewares\":{\"redirect-web-to-websecure\":{\"redirectScheme\":{\"scheme\":\"https\",\"port\":\"443\",\"permanent\":true}}}},\"tcp\":{},\"tls\":{}}" providerName=internal
time="2020-07-26T17:01:04Z" level=info msg="Starting provider *acme.Provider {\"email\":\"[email protected]\",\"caServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"storage\":\"/data/acme.json\",\"keyType\":\"RSA4096\",\"tlsChallenge\":{},\"ResolverName\":\"letsencrypt\",\"store\":{},\"ChallengeStore\":{}}"
time="2020-07-26T17:01:04Z" level=info msg="Testing certificate renew..." providerName=letsencrypt.acme
time="2020-07-26T17:01:04Z" level=debug msg="Added outgoing tracing middleware noop@internal" entryPointName=web routerName=web-to-websecure@internal middlewareName=tracing middlewareType=TracingForwarder
time="2020-07-26T17:01:04Z" level=debug msg="Configuration received from provider letsencrypt.acme: {\"http\":{},\"tls\":{}}" providerName=letsencrypt.acme
time="2020-07-26T17:01:04Z" level=debug msg="Creating middleware" middlewareName=redirect-web-to-websecure@internal middlewareType=RedirectScheme routerName=web-to-websecure@internal entryPointName=web
time="2020-07-26T17:01:04Z" level=debug msg="Setting up redirection to https 443" middlewareType=RedirectScheme routerName=web-to-websecure@internal entryPointName=web middlewareName=redirect-web-to-websecure@internal
time="2020-07-26T17:01:04Z" level=debug msg="Adding tracing to middleware" entryPointName=web routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal
time="2020-07-26T17:01:04Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2020-07-26T17:01:04Z" level=debug msg="No default certificate, generating one"
E0726 17:01:04.892814       1 reflector.go:178] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:125: Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
E0726 17:01:04.896024       1 reflector.go:178] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:125: Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
time="2020-07-26T17:01:05Z" level=debug msg="Added outgoing tracing middleware noop@internal" routerName=web-to-websecure@internal entryPointName=web middlewareName=tracing middlewareType=TracingForwarder
time="2020-07-26T17:01:05Z" level=debug msg="Creating middleware" routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal middlewareType=RedirectScheme entryPointName=web
time="2020-07-26T17:01:05Z" level=debug msg="Setting up redirection to https 443" routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal middlewareType=RedirectScheme entryPointName=web
time="2020-07-26T17:01:05Z" level=debug msg="Adding tracing to middleware" entryPointName=web routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal
time="2020-07-26T17:01:05Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=web
time="2020-07-26T17:01:05Z" level=debug msg="No default certificate, generating one"
E0726 17:01:08.006765       1 reflector.go:178] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:125: Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
E0726 17:01:12.311744       1 reflector.go:178] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:125: Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
E0726 17:01:23.452737       1 reflector.go:178] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:125: Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
E0726 17:01:39.526007       1 reflector.go:178] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:125: Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
E0726 17:02:16.043578       1 reflector.go:178] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:125: Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope

RBAC

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: default

Traefik Ingress

kind: Ingress
apiVersion: networking.k8s.io/v1beta1
metadata:
  name: myingress
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls.certresolver: letsencrypt
    traefik.ingress.kubernetes.io/router.tls.domains.0.main: traefik.example.in
spec:
  rules:
    - host: traefik.example.in
      http:
        paths:
          - path: /
            backend:
              serviceName: traefik
              servicePort: 8080
     

Deployment Traefik

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: default
  name: traefik-ingress-controller

---
### Deploy Traefik to a Cluster ###
## We can use Deployment, DaemonSet or Helm Chart
kind: Deployment
apiVersion: apps/v1
metadata:
  namespace: default
  name: traefik
  labels:
    app: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      labels:
        app: traefik
    spec:
      tolerations:
      - effect: NoSchedule
        operator: Exists
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - name: traefik
        image: traefik:v2.3
        imagePullPolicy: IfNotPresent
        resources:
            limits:
              memory: 400Mi
              cpu: 400m
            requests:
              memory: 400Mi
              cpu: 400m
        args:
        - --log=true
        - --log.level=DEBUG
        - --accesslog
        #- --providers.kubernetescrd # use this when using IngressRoute
        - --providers.kubernetesingress # use this when using Ingress
        - --entryPoints.web.address=:80
        # - --entrypoints.web.http.redirections.entryPoint.to=websecure
        # - --entrypoints.web.http.redirections.entryPoint.scheme=https
        - --entryPoints.websecure.address=:443
        - --certificatesResolvers.letsencrypt.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory
        - --certificatesResolvers.letsencrypt.acme.tlsChallenge
        - [email protected]
        - --certificatesResolvers.letsencrypt.acme.storage=/data/acme.json        
        ports:
        - name: web
          containerPort: 80
        - name: admin
          containerPort: 8080
        - name: websecure
          containerPort: 443  
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        volumeMounts:
          - mountPath: /data
            name: storage-volume    

      restartPolicy: Always
      volumes:
        - name: storage-volume
          persistentVolumeClaim:
              claimName: traefik-acme-storage

Service Traefik

apiVersion: v1
kind: Service
metadata:
  name: traefik
spec:
  type: LoadBalancer
  selector:
    app: traefik
  ports:
    - protocol: TCP
      port: 80
      name: web
      targetPort: 80
    - protocol: TCP
      port: 8080
      name: admin
      targetPort: 8080

Please help. I'm new to Kubernetes. I'm already using Traefik with docker-swarm but there's a lot difference how we use Traefik with K8s and with docker.

Answer 1

try this in your ClusterRole:

  - apiGroups:
      - extensions
      - networking.k8s.io
    resources:
      - ingresses
      - ingressclasses
    verbs:
      - get
      - list
      - watch

instead of your

  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch

That worked for me.