GAE AttributeError: 'Credentials' object has no attribute 'with_subject'

Question

I have a python app I want to deploy on App Engine (2nd Generation Python 3.7) on which I use a Service Account with Domain-wide delegation enabled to access user data.

Locally I do:

import google.auth
from apiclient.discovery import build

creds, project = google.auth.default(
    scopes=['https://www.googleapis.com/auth/admin.directory.user', ],
)
creds = creds.with_subject(GSUITE_ADMIN_USER)

service = build('admin', 'directory_v1', credentials=creds)

This works good and, as far as I know it is the current way to do this when using Application Default Credentials (locally I have GOOGLE_APPLICATION_CREDENTIALS defined).

Problem is on GAE, when deployed, the call to with_subject raises: AttributeError: 'Credentials' object has no attribute 'with_subject'

I have enabled Domain-wide delegation on the GAE service account already.

What is different between the GOOGLE_APPLICATION_CREDENTIALS I use locally and the ones in GAE when both are service accounts with domain-wide delegation?

Where is .with_subject() on GAE?

The creds object received is of type compute_engine.credentials.Credentials.

Full traceback:

Traceback (most recent call last):
  File "/env/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 104, in init_process
    super(ThreadWorker, self).init_process()
  File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/env/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/env/lib/python3.7/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/srv/main.py", line 1, in <module>
    from config.wsgi import application
  File "/srv/config/wsgi.py", line 38, in <module>
    call_command('gsuite_sync_users')
  File "/env/lib/python3.7/site-packages/django/core/management/__init__.py", line 148, in call_command
    return command.execute(*args, **defaults)
  File "/env/lib/python3.7/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/srv/metanube_i4/users/management/commands/gsuite_sync_users.py", line 14, in handle
    gsuite_sync_users()
  File "/env/lib/python3.7/site-packages/celery/local.py", line 191, in __call__
    return self._get_current_object()(*a, **kw)
  File "/env/lib/python3.7/site-packages/celery/app/task.py", line 375, in __call__
    return self.run(*args, **kwargs)
  File "/srv/metanube_i4/users/tasks.py", line 22, in gsuite_sync_users
    creds = creds.with_subject(settings.GSUITE_ADMIN_USER)
AttributeError: 'Credentials' object has no attribute 'with_subject'"  

Packages (partial list):

google-api-core==1.5.0
google-api-python-client==1.7.4
google-auth==1.5.1
google-auth-httplib2==0.0.3
google-cloud-bigquery==1.6.0
google-cloud-core==0.28.1
google-cloud-logging==1.8.0
google-cloud-storage==1.13.0
google-resumable-media==0.3.1
googleapis-common-protos==1.5.3
httplib2==0.11.3
oauthlib==2.1.0

Answer 1

It is true that you cannot use the with_subject method with GAE or GCE credentials. However, there is a workaround that I was able to get working on my GCE server and I would assume this works with GAE default service accounts as well. The solution is to build new credentials using the service account identity with desired subject and scopes. A detailed guide can be found here, but I will also explain the process bellow.

Firstly, the service account needs permissions to create service account tokens for itself. This can be done by going to the projects IAM and admin > Service accounts page (make sure the info panel is visible, it can be toggled from the top right corner). Copy the service account email address and select the service account in question by ticking the checkbox. Now the info panel should have ADD MEMBER button. Click it and paste the service account email to the New members textbox. Click the Select role dropdown and choose the role Service Accounts -> Service Account Token Creator. You can check that the role is assigned with the following gcloud command:

gcloud iam service-accounts get-iam-policy [SERVICE_ACCOUNT_EMAIL]

Now to the actual Python code. This example is a slight modification from the documentation linked above.

from googleapiclient.discovery import build
from google.auth import default, iam
from google.auth.transport import requests
from google.oauth2 import service_account

TOKEN_URI = 'https://accounts.google.com/o/oauth2/token'
SCOPES = ['https://www.googleapis.com/auth/admin.directory.user']
GSUITE_ADMIN_USER = '[email protected]'

def delegated_credentials(credentials, subject, scopes):
    try:
        # If we are using service account credentials from json file
        # this will work
        updated_credentials = credentials.with_subject(subject).with_scopes(scopes)
    except AttributeError:
        # This exception is raised if we are using GCE default credentials

        request = requests.Request()

        # Refresh the default credentials. This ensures that the information
        # about this account, notably the email, is populated.
        credentials.refresh(request)

        # Create an IAM signer using the default credentials.
        signer = iam.Signer(
            request,
            credentials,
            credentials.service_account_email
        )

        # Create OAuth 2.0 Service Account credentials using the IAM-based
        # signer and the bootstrap_credential's service account email.
        updated_credentials = service_account.Credentials(
            signer,
            credentials.service_account_email,
            TOKEN_URI,
            scopes=scopes,
            subject=subject
        )
    except Exception:
        raise

    return updated_credentials


creds, project = default()
creds = delegated_credentials(creds, GSUITE_ADMIN_USER, SCOPES) 

service = build('admin', 'directory_v1', credentials=creds)

The try block will not fail if you have GOOGLE_APPLICATION_CREDENTIALS environment variable set with a path to a service account file. If the application is run on Google Cloud, there will be an AttributeError and it is handled by creating new credentials which have correct subject and scopes.

You can also pass None as the subject for delegated_credentials function and it creates the credentials without delegation so this function can be used with or without delegation.