How to fix Flannel CNI plugin. Error: [plugin flannel does not support config version ""]

Question

While installing Kubernetes with, I'm stuck at CNI plugin installation and configuration part. I have installed Flannel but I see error in kubelet logs due to which coredns pods are in pending state.

OS: Centos7 k8s version: 1.16 Kubeadm is being used to setup the cluster.

I had installed the plugin using: kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

This is error I can see in Kubelet logs:

Sep 21 04:47:29 peteelizalde2c kubelet: W0921 04:47:29.897759   17817 cni.go:202] Error validating CNI config &{cbr0  false [0xc000fb3ee0 0xc000fb3f60] [123 10 32 32 34 110 97 109 101 34 58 32 34 99 98 114 48 34 44 10 32 32 34 112 108 117 103 105 110 115 34 58 32 91 10 32 32 32 32 123 10 32 32 32 32 32 32 34 116 121 112 101 34 58 32 34 102 108 97 110 110 101 108 34 44 10 32 32 32 32 32 32 34 100 101 108 101 103 97 116 101 34 58 32 123 10 32 32 32 32 32 32 32 32 34 104 97 105 114 112 105 110 77 111 100 101 34 58 32 116 114 117 101 44 10 32 32 32 32 32 32 32 32 34 105 115 68 101 102 97 117 108 116 71 97 116 101 119 97 121 34 58 32 116 114 117 101 10 32 32 32 32 32 32 125 10 32 32 32 32 125 44 10 32 32 32 32 123 10 32 32 32 32 32 32 34 116 121 112 101 34 58 32 34 112 111 114 116 109 97 112 34 44 10 32 32 32 32 32 32 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 32 123 10 32 32 32 32 32 32 32 32 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 32 116 114 117 101 10 32 32 32 32 32 32 125 10 32 32 32 32 125 10 32 32 93 10 125 10]}: [plugin flannel does not support config version ""]
Sep 21 04:47:29 peteelizalde2c kubelet: W0921 04:47:29.897824   17817 cni.go:237] Unable to update cni config: no valid networks found in /etc/cni/net.d
Sep 21 04:47:32 peteelizalde2c kubelet: E0921 04:47:32.007379   17817 kubelet.go:2187] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

Here is the pods:

kubectl get pods --all-namespaces
NAMESPACE     NAME                                                                        READY   STATUS    RESTARTS   AGE
kube-system   coredns-5644d7b6d9-n4h5x                                                    0/1     Pending   0          114m
kube-system   coredns-5644d7b6d9-t2q54                                                    0/1     Pending   0          114m
kube-system   etcd-ip-10-29-89-124                      1/1     Running   0          113m
kube-system   kube-apiserver-ip-10-29-89-124            1/1     Running   0          113m
kube-system   kube-controller-manager-ip-10-29-89-124   1/1     Running   0          113m
kube-system   kube-flannel-ds-amd64-dqpzj                                                 1/1     Running   0          110m
kube-system   kube-proxy-vzlqb                                                            1/1     Running   0          114m
kube-system   kube-scheduler-ip-10-29-89-124            1/1     Running   0          113m

There is a file in /etc/cni/net.d named 10-flannel.conflist. Its contents are:

{
  "name": "cbr0",
  "plugins": [
    {
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}

Answer 1

The accepted solution is incomplete and will cause headache down the line.

The proper solution to make this change permanent is to edit the ConfigMap created by flannel in your Kubernetes cluster. Otherwise, the file will be recreated the next time the flannel pod volumes are populated with the ConfigMap (e.g. on node reboot).

Use kubectl edit cm -n kube-system kube-flannel-cfg to edit the ConfigMap provided by flannel, and add the missing line:

  5 apiVersion: v1
  6 data:
  7   cni-conf.json: |
  8     {
  9       "name": "cbr0",
 10       "cniVersion": "0.2.0",
 11       "plugins": [

Reboot the node, or alternatively make the change manually in /etc/cni/net.d/10-flannel.conflist and do systemctl restart kubelet afterwards to skip the reboot.

Answer 2

The file /etc/cni/net.d/10-flannel.conflist was missing cniVersion key in its config.

Adding "cniVersion": "0.2.0" solved the issue.