DigitalOcean pod has unbound immediate PersistentVolumeClaims

Question

I am trying to run a Redis cluster in Kubernetes in DigitalOcean. As a poc, I simply tried running an example I found online (https://github.com/sanderploegsma/redis-cluster/blob/master/redis-cluster.yml), which is able to spin up the pods appropriately when running locally using minikube.

However, when running it on Digital Ocean, I always get the following error:

Warning FailedScheduling 3s (x8 over 17s) default-scheduler pod has unbound immediate PersistentVolumeClaims (repeated 4 times)

Given that I am not changing anything, I am not sure why this would not work. Does anyone have any suggestions?

EDIT: some additional info

$ kubectl describe pvc  
Name:          data-redis-cluster-0
Namespace:     default
StorageClass:  
Status:        Pending
Volume:        
Labels:        app=redis-cluster
Annotations:   <none>
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
Events:
  Type       Reason         Age                     From                         Message
  ----       ------         ----                    ----                         -------
  Normal     FailedBinding  3m19s (x3420 over 14h)  persistentvolume-controller  no persistent volumes available for this claim and no storage class is set
Mounted By:  <none>

EDIT: setting the default storage class partially resolved the problem! However, the node is now not able to find available volumes to bind:

kubectl describe pvc:

Name:          data-redis-cluster-0
Namespace:     default
StorageClass:  local-storage
Status:        Pending
Volume:        
Labels:        app=redis-cluster
Annotations:   <none>
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
Events:
  Type       Reason                Age                     From                         Message
  ----       ------                ----                    ----                         -------
  Normal     WaitForFirstConsumer  12m (x9 over 13m)       persistentvolume-controller  waiting for first consumer to be created before binding
  Normal     WaitForFirstConsumer  3m19s (x26 over 9m34s)  persistentvolume-controller  waiting for first consumer to be created before binding

kubectl describe pod redis-cluster-0

....
Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  16m (x25 over 17m)   default-scheduler  0/5 nodes are available: 1 node(s) had taints that the pod didn't tolerate, 4 node(s) didn't find available persistent volumes to bind.

kubectl describe sc

Name:                  local-storage
IsDefaultClass:        Yes
Annotations:           storageclass.kubernetes.io/is-default-class=true
Provisioner:           kubernetes.io/no-provisioner
Parameters:            <none>
AllowVolumeExpansion:  <unset>
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     WaitForFirstConsumer
Events:                <none>

kubernetes manager pod logs:

I1028 15:30:56.154131       1 event.go:221] Event(v1.ObjectReference{Kind:"StatefulSet", Namespace:"default", Name:"redis-cluster", UID:"7528483e-dac6-11e8-871f-2e55450d570e", APIVersion:"apps/v1", ResourceVersion:"2588806", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' create Claim data-redis-cluster-0 Pod redis-cluster-0 in StatefulSet redis-cluster success
I1028 15:30:56.166649       1 event.go:221] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"data-redis-cluster-0", UID:"76746506-dac6-11e8-871f-2e55450d570e", APIVersion:"v1", ResourceVersion:"2588816", FieldPath:""}): type: 'Normal' reason: 'WaitForFirstConsumer' waiting for first consumer to be created before binding
I1028 15:30:56.220464       1 event.go:221] Event(v1.ObjectReference{Kind:"StatefulSet", Namespace:"default", Name:"redis-cluster", UID:"7528483e-dac6-11e8-871f-2e55450d570e", APIVersion:"apps/v1", ResourceVersion:"2588806", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' create Pod redis-cluster-0 in StatefulSet redis-cluster successful
I1028 15:30:57.004631       1 event.go:221] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"data-redis-cluster-0", UID:"76746506-dac6-11e8-871f-2e55450d570e", APIVersion:"v1", ResourceVersion:"2588825", FieldPath:""}): type: 'Normal' reason: 'WaitForFirstConsumer' waiting for first consumer to be created before binding

Answer 1

This:

no storage class is set

And an empty output for kubectl describe sc means that there's no storage class.

I recommend installing the CSI-driver for Digital Ocean. That will create a do-block-storage class using the Kubernetes CSI interface.

Another option is to use local storage. Using a local storage class:

$ cat <<EOF
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
EOF | kubectl apply -f -

Then for either case you may need to set it as a default storage class if you don't specify storageClassName in your PVC:

$ kubectl patch storageclass local-storage -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

or

$ kubectl patch storageclass do-block-storage -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Answer 2

It is a statefulSet using PersistentVolumeClaims

You need to configure a default storageClass in your cluster so that the PersistentVolumeClaim can take the storage from there.

In minikube one is already available so it succeeds without error:

C02W84XMHTD5:ucp iahmad$ kubectl get sc --all-namespaces 
NAME                 PROVISIONER                AGE
standard (default)   k8s.io/minikube-hostpath   7d