Cloud composer: "PERMISSION_DENIED: The caller does not have permission"

Question

I implemented a few tasks with BashOperator. Ones with "gsutil rm" and "gsutil cp" worked fine. But one with "gcloud alpha firestore export" generates this error:

{bash_operator.py:101} INFO - ERROR: (gcloud.alpha.firestore.export) PERMISSION_DENIED: The caller does not have permission

This command itself works fine in gcloud shell. I tried to give some Firestore related permissions to the service account used by the Composer but it still doesn't work. Any idea

Answer 1

It might be that you don't have permissions for a particular project.

The error I was getting was: PERMISSION_DENIED: Caller does not have required permission to use project project:random-id-11111.

The way I resolved it was by running gcloud config set project 'the-right-project-id' and then the actual gcloud command.

Answer 2

I think you need Cloud Datastore Import Export access. Following are the steps as per current Current Google Cloud platform layout.

https://console.cloud.google.com > Left drawer > IAM & admin > Against user - Edit Icon > Add another role > Data Store > Cloud Datastore Import & Export > Save