Using standalone 'gsutil' from within GKE

Question

I'm trying to use the standalone gsutil tool from within a container running in a GKE cluster, but I cannot get it to work. I believe the cluster has adequate permissions (see below). However, running

./gsutil ls gs://my-bucket/

yields

ServiceException: 401 Anonymous users does not have storage.objects.list access to bucket my-bucket.

Am I missing anything? I don't have a .boto file, as I believe it shouldn't be necessary—or is it? This is the list of scopes that the cluster and the node pool have:

- https://www.googleapis.com/auth/compute
- https://www.googleapis.com/auth/devstorage.full_control
- https://www.googleapis.com/auth/logging.write
- https://www.googleapis.com/auth/monitoring.write
- https://www.googleapis.com/auth/pubsub
- https://www.googleapis.com/auth/servicecontrol
- https://www.googleapis.com/auth/service.management.readonly
- https://www.googleapis.com/auth/trace.append

Answer 1

You can use gsutil inside a docker container on GKE with a service account, or with your own credentials.

Service Account

1) Add the service-account.json file to your project.

2) Add a .boto file to your project pointing to the service-account.json file:

[Credentials]
gs_service_key_file = /path/to/service-account.json

3) In your Dockerfile, set the BOTO_CONFIG environment variable to point to this .boto file:

ENV BOTO_CONFIG=/path/to/.boto

Own Credentials

1) Locally, run gcloud auth login. A .boto file will be created at ~/.config/gcloud/legacy_credentials/[email protected]/.boto with the following structure:

[OAuth2]
client_id = <id>.apps.googleusercontent.com
client_secret = <secret>

[Credentials]
gs_oauth2_refresh_token = <token>

2) Copy this .boto file into your project

3) In your Dockerfile, set the BOTO_CONFIG environment variable to point to this .boto file:

ENV BOTO_CONFIG=/path/to/.boto

I installed standalone gsutil in the docker container using pip install gsutil