403 "Request had insufficient authentication scopes" during gcloud container cluster get-credentials

Question

From a VM in GCE, I did the following

gcloud auth activate-service-account --key-file <blah>
# "blah" is a service account key file (JSON) I generated from the web interface
gcloud config set project <project-name>
gcloud config set compute/zone <zone-name>
gcloud set container/cluster <cluster-name>

Then when I tried to run

gcloud container clusters get-credentials <cluster-name>

and it failed with the error message:

Error message: "ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Request had insufficient authentication scopes."

The VM is on the same network as the GKE cluster. I tried the same thing, with the same service account key file from a machine outside GCE, against a GKE cluster on the "default" network and it succeeded...

Answer 1

To use the Google Kubernetes Engine API from a GCE virtual machine you need to add the cloud platform scope ("https://www.googleapis.com/auth/cloud-platform") to your VM when it is created.

Answer 2

There is now a solution (in beta and alpha only) to set scope on an existing GCE VM. All it needs to be successful is to stop the VM before executing the command.

First, you should be aware (and copy) current scopes of your VM so you can set them along with your new scopes, use:

gcloud compute instances describe your-instance

At the bottom you should see a list of scopes, copy them.

Then, read documentation for this command in beta (available to everyone but to be used at your own risk): https://cloud.google.com/sdk/gcloud/reference/beta/compute/instances/set-scopes

Before you execute this command, stop the instance from the GCE page and wait for it to be shut down. A scary warning will appear, be aware that if the VM does not shut down gracefully in 90 seconds (= all processes and services successfully turned off) the file system might get corrupt when force shutting down the VM. Take good note and backup important files if you feel unsafe about this.

For me, with the existing scopes plus the new one (cloud-platform) the resulting set-scopes command was:

gcloud beta compute instances set-scopes my-instance --zone=us-central1-a --scopes=https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/trace.append,https://www.googleapis.com/auth/cloud-platform 

Answer 3

If you are using The Google Kubernetes Engine API from a VM in GCP. You first need to add required scope at vm level https://www.googleapis.com/auth/projecthosting,cloud-platform. This can be done by GCP console as well. Stop the VM instance then go to edit option and at the end you will find Cloud API access scopes.

Answer 4

Step 1 : gcloud init

Step 2 : Select [2] Create a new configuration

Step 3 : Enter configuration name. Names start with a lower case letter and contain only lower case letters a-z, digits 0-9, and hyphens '-': kubernetes-service-account

Step 4 : Choose the account you would like to use to perform operations for this configuration:[2] Log in with a new account

Step 5 : Do you want to continue (Y/n)? y

Step 6 : Copy paste the link to brwoser and login with the ID which is used to create your google Cloud Account

Step 7 : Copy the verification code provided by google after login and paste it in to the console.

Step 8 : Pick cloud project to use:

Step 9: Do you want to configure a default Compute Region and Zone? (Y/n)? y

Step 10 : Please enter numeric choice or text value (must exactly match list item): 8

Your Google Cloud SDK is configured and ready to use!

Once this is done, make sure the service account configured for the VM has permissions to do the required tasks.