Volume claim on GKE / Multi-Attach error for volume Volume is already exclusively attached

Question

The problem seems to have been solved a long time ago, as the answer and the comments does not provide real solutions, I would like to get some help from experienced users

The error is the following (when describing the pod, which keeps on the ContainerCreating state) :

Multi-Attach error for volume "pvc-xxx" Volume is already exclusively attached to one node and can't be attached to another 

This all run on GKE. I had a previous cluster, and the problem never occured. I have reused the same disk when creating this new cluster — not sure if it is related

Here is the full yaml config files (I'm leaving the concerned code part commented as to highlight it; it is not when in effective use)

Thanks in advance if obvious workarounds

Answer 1

Based on your description what you are experiencing is exactly what is supposed to happen.

You are using gcePersistentDisk in your PV/PVC definition. The accessMode is ReadWriteOnce - this means that this PV can only be attached to a single Node (stressing Node here, there can be multiple Pods running on the same Node using the same PV). There is not much you can do about this; gcePersistentDisk is like a remote block device, it's not possible to mount it on multiple Nodes simultaneously (unless read only).

There is a nice table that shows which PVs support ReadWriteMany (that is, write access on multiple Nodes at the same time):

Important! A volume can only be mounted using one access mode at a time, even if it supports many. For example, a GCEPersistentDisk can be mounted as ReadWriteOnce by a single node or ReadOnlyMany by many nodes, but not at the same time.

Answer 2

Your Deployment yaml shows 5 replicas, which will not work with GCE PD in ReadWriteOnce mode. GCE PD can only be attached to multiple nodes in ReadOnlyMany mode.

If you need shared, writable storage across all replicas, then you should look into a multi-writer solution like NFS or Gluster.

If you want each replica to have its own disk, then you can use StatefulSets, which will have a PVC per replica.