Kubernetes : Dynamic Persistent Volume provisioning using NFS

Question

I have multi node kubernetes setup. I am trying to allocate a Persistent volume dynamically using storage classes with NFS volume plugin. I found storage classes examples for glusterfs, aws-ebs, etc.but, I didn't find any example for NFS. If I create PV and PVC only then NFS works very well(Without storage class). I tried to write storage class file for NFS, by referring other plugins. please refer it below,

nfs-storage-class.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  namespace: kube-system
  name: my-storage
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
  labels:
    kubernetes.io/cluster-service: "true"

provisioner: kubernetes.io/nfs
parameters:
  path: /nfsfileshare
  server: <nfs-server-ip> 

nfs-pv-claim.yaml

apiVersion: v1
metadata:
  name: demo-claim
  annotations:
    volume.beta.kubernetes.io/storage-class: my-storage
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi

It didn't worked. So, my question is, Can we write a storage class for NFS? Does it support dynamic provisioing?

Answer 1

As of August 2020, here's how things look for NFS persistence on Kubernetes:

You can

• Put an NFS volume on a Pod directly:

apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: k8s.gcr.io/test-webserver
    name: test-container
    volumeMounts:
    - mountPath: /test-pd
      name: test-volume
  volumes:
  - name: test-volume
    nfs:
      path: /foo/bar
      server: wherever.dns

• Manually create a Persistent Volume backed by NFS, and mount it with a Persistent Volume Claim (PV spec shown below):

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0003
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: slow
  mountOptions:
    - hard
    - nfsvers=4.1
  nfs:
    path: /tmp
    server: 172.17.0.2

• Use the (now deprecated) NFS PV provisioner from external-storage. This was last updated two years ago, and has been officially EOL'd, so good luck. With this route, you can make a Storage Class such as the one below to fulfill PVCs from your NFS server.
  • Update: There is a new incarnation of this provisioner as kubernetes-sigs/nfs-subdir-external-provisioner! It seems to work in a similar way to the old nfs-client provisioner, but is much more up-to-date. Huzzah!

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: example-nfs
provisioner: example.com/nfs
mountOptions:
  - vers=4.1

• Evidently, CSI is the future, and there is a NFS CSI driver. However, it doesn't support dynamic provisioning yet, so it's not really terribly useful.
  • Update (December 2020): Dynamic provisioning is apparently in the works (on master, but not released) for the CSI driver.
• You might be able to replace external-storage's NFS provisioner with something from the community, or something you write. In researching this problem, I stumbled on a provisioner written by someone on GitHub, for example. Whether such provisioners perform well, are secure, or work at all is beyond me, but they do exist.

Answer 2

I'm looking into doing the same thing. I found https://github.com/kubernetes-incubator/external-storage/tree/master/nfs, which I think you based your provisioner on?

I think an nfs provider would need to create a unique directory under the path defined. I'm not really sure how this could be done.

Maybe this is better of as an github issue on the kubernetes repo.

Answer 3

Dynamic storage provisioning using NFS doesn't work, better use glusterfs. There's a good tutorial with fixed to common problems while setting up. http://blog.lwolf.org/post/how-i-deployed-glusterfs-cluster-to-kubernetes/