Permissions error when attaching Azure Disk to AKS pod

Question

I've been battling this error for a few hours now. Found several articles but nothing that has helped so far.

My work has been based on the "How-to guide > Configure data volumes > Azure Disk - Static" from https://learn.microsoft.com/en-us/azure/aks/azure-disk-volume. As far as I can tell, I've not missed any steps.

I have defined by config in a YAML file as so:

kind: Deployment
metadata:
    name: phio-dep-db
    namespace: production
spec:
    selector:
        matchLabels:
            app: phio
    replicas: 1
    strategy:
        type: RollingUpdate
    template:
        metadata:
            labels:
                app: phio
        spec:
            terminationGracePeriodSeconds: 1
            volumes:
              - name: dbvolume
                azureDisk:
                    diskName: PHIO_DB_VOL
                    diskURI: /subscriptions/<my-sub-id/resourceGroups/<my-grp>/providers/Microsoft.Compute/disks/PHIO_DB_VOL
                    kind: Managed
            containers:
              - image: phioweb.azurecr.io/phiodb
                name: db
                ports:
                  - containerPort: 3306
                volumeMounts:
                  - mountPath: /var/lib/mysql
                    name: dbvolume

However, when I do kubectl apply -f .yml, the pod never starts. If I do a "kubectl describe all -n production" I see the following:

  Type     Reason              Age                 From                                        Message
  ----     ------              ----                ----                                        -------
  Normal   Scheduled           31m                 default-scheduler                           Successfully assigned production/phio-dep-db-6c6ddc885-p4ntg to aks-nodepool1-18816840-vmss000001
  Warning  FailedAttachVolume  79s (x23 over 31m)  attachdetach-controller                     AttachVolume.Attach failed for volume "dbvolume" : compute.DisksClient#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '<GUID>' with object id '<same GUID>' does not have authorization to perform action 'Microsoft.Compute/disks/read' over scope '/subscriptions/<my-sub-id/resourceGroups/<my-grp>/providers/Microsoft.Compute/disks/PHIO_DB_VOL' or the scope is invalid. If access was recently granted, please refresh your credentials."
  Warning  FailedMount         23s (x14 over 29m)  kubelet, aks-nodepool1-18816840-vmss000001  Unable to mount volumes for pod "phio-dep-db-6c6ddc885-p4ntg_production(0e326781-2936-4451-9446-9608a8f41439)": timeout expired waiting for volumes to attach or mount for pod "production"/"phio-dep-db-6c6ddc885-p4ntg". list of unmounted volumes=[dbvolume]. list of unattached volumes=[dbvolume default-token-s7svk]

There is nothing in the MS documentation about assigning permissions to the Azure Disk, and there are no apparent parameters I'm missing in the YAML.

Anyone got any ideas what I'm missing?

Many thanks.

(I've sanitized the output a little, but it's all test and the Azure subscription will be binned after this work, even though it says production)

Answer 1

From the docs here

When you create an Azure disk for use with AKS, you can create the disk resource in the node resource group. This approach allows the AKS cluster to access and manage the disk resource.If you instead create the disk in a separate resource group, you must grant the Azure Kubernetes Service (AKS) service principal for your cluster the Contributor role to the disk's resource group. Alternatively, you can use the system assigned managed identity for permissions instead of the service principal. For more information, see Use managed identities

From the docs here

You may need to access existing Disk resources in another resource group. Assign one of the following set of role permissions:

1. Create a custom role and define the following role permissions: Microsoft.Compute/disks/read Microsoft.Compute/disks/write
2. Or, assign the Storage Account Contributor built-in role on the resource group