"DatastoreException: Missing or insufficient permissions" on superseded Cloud Datastore

Question

We have an older application based on the AppEngine SDK (now deprecated) and the superseded Cloud Datastore. In the process of migrating to Google Cloud SDK we also decided to move from JPA/Datanucleus to Objectify.

Given that the Cloud Datastore will be automatically upgraded to Cloud Firestore in Datastore mode sometime in the future, we decided to test our application as described at the bottom of this page: https://cloud.google.com/datastore/docs/upgrade-to-firestore#testing_an_existing_application

1) Create a new project. In this project, create a Cloud Firestore in Datastore mode database.

2) Using the managed export service, export some of your application's data to Cloud Storage.

3) Using the managed import service, import your application's data to your new project.

4) Copy app logic you want to test to the new project or simulate app behaviour against the new project.

That's what we did and after some issues we could make a portion of our application run fine with the new datastore in a separate test project.

Now to the actual issue...

We wanted to test if the updated application could also run with the superseded Cloud Datastore, so we won't have to worry when the automatic upgrade occurs (as our app will be already ready). So we deployed it as a new version of the existing AppEngine project (v2-dot-.....): unfortunately running the new version throws a permission error as soon as the app tries to read the datastore:

com.google.cloud.datastore.DatastoreException: Missing or insufficient permissions

So the questions are: - could this be related to the Cloud Datastore not being upgraded to Cloud Firestore in Datastore mode for our project yet? - is there anything we can do (add specific permissions maybe) to make it work anyway?

Our concern is that we need to have the new version of the app deployed before July 2020 (that's when the older AppEngine SDK will stop working), and we are worried that the automatic upgrade of the datastore will occur later.

Thank you for your help.

Answer 1

Turned out our project doesn't have the [email protected] member in IAM.

It has a pletora of other members (for example: [email protected], [email protected], [email protected], etc) which I guess are legacy members used in previous versions of Google App Engine.

Adding [email protected] with the role Editor fixed the issue: now the new version can be deployed to the old projects and it works fine even if the datastore has not yet been converted to Cloud Firestore in Datastore Mode.

Answer 2

I just ran into this issue and spent way too much time troubleshooting it. Nine times out of ten if you're running into this issue it's because the default App Engine service account doesn't have permission to edit Cloud Datastore. The default App Engine service account is used by default if you're doing a simple gcloud app deploy and nothing else fancy. I solved the problem by giving the default App Engine service account the roles/datastore.owner role with the following:

gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="serviceAccount:[email protected]" \
    --role="roles/datastore.owner"