Accessing GAE log files using Google Cloud Logging (Python)

Question

We have a running Google App Engine (GAE) service for which we would like to download the logs for archival on our server.

The GAE has a Service Account, the Credentials for which have been downloaded as a JSON file to our server. The following code, run on our server, attempts to create a client for the logging service:

from google.cloud import logging
client = logging.Client.from_service_account_json('credentials.json')

with the result:

ValueError: Service account info was not in the expected format, missing fields token_uri, client_email.

The error message is quite clear, but what is not clear is why the fields are expected in a JSON file that was created for this purpose? Are we using the credentials from the wrong type of Service account?

Answer 1

You need to get the service account file that contains the private key credentials, it's basically a different file from the one you have. You can get it, or get a new one by going to https://console.developers.google.com/iam-admin/iam/ then select your project, then select "Service accounts" and create a new one as role "viewer" for the project for example (or use one that already exists and click "create new key")

The "key" is a json or p12 file that will be downloaded when you create the account (or use "create new key" there) which contains the correct fields and credentials that will work for your code.

Example structure of the downloaded "key" file (when selecting JSON):

{
  "type": "service_account",
  "project_id": "zeta-handler-9999",
  "private_key_id": "123456789deedbeaf",
  "private_key": "-----BEGIN PRIVATE KEY-----\nREDACTED REDACTED...-----END PRIVATE KEY-----\n",
  "client_email": "projectname-service-account@zeta-handler-9999.iam.gserviceaccount.com",
  "client_id": "12345678909999",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/projectname-service-account%40zeta-handler-9999.iam.gserviceaccount.com"
}

Example code to use that "key" file (python):

#!/usr/bin/env python
import google.auth
from google.oauth2 import service_account

credentials = service_account.Credentials.from_service_account_file('downloaded_key.json')
scoped_credentials = credentials.with_scopes(['https://www.googleapis.com/auth/drive.metadata.readonly'])