Appengine remote_api_shell not working with application-default credentials since update

Question

I recently updated my gcloud libraries from 118.0.0 to 132.0.0 and immediately remote_api_shell no longer worked. I went through a number of permutations of re-logging in, to set the application-default credentials through gcloud, and to use a service account and environment variable. All permutations failed with the same error message:

    Traceback (most recent call last):
  File "/Users/mbostwick/google-cloud-sdk/bin/remote_api_shell.py", line 133, in <module>
    run_file(__file__, globals())
  File "/Users/mbostwick/google-cloud-sdk/bin/remote_api_shell.py", line 129, in run_file
    execfile(_PATHS.script_file(script_name), globals_)
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 160, in <module>
    main(sys.argv)
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 156, in main
    oauth2=True)
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 74, in remote_api_shell
    secure=secure, app_id=appid)
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 769, in ConfigureRemoteApiForOAuth
    rpc_server_factory=rpc_server_factory)
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 839, in ConfigureRemoteApi
    app_id = GetRemoteAppIdFromServer(server, path, rtok)
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 569, in GetRemoteAppIdFromServer
    response = server.Send(path, payload=None, **urlargs)
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 259, in Send
    NeedAuth()
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 235, in NeedAuth
    RaiseHttpError(url, response_info, response, 'Too many auth attempts.')
  File "/Users/mbostwick/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 85, in RaiseHttpError
    raise urllib2.HTTPError(url, response_info.status, msg, response_info, stream)
urllib2.HTTPError: HTTP Error 401: Unauthorized Too many auth attempts.

After back revving through 131.0.0 and 130.0.0, I just went back to 118.0.0, re-logged in and everything worked fine.

I did not update the running application after updating gcloud, as I'm in the middle of a release cycle at the moment, so that may have been the issue, but any help would be appreciated. Thanks!

Answer 1

TL;DR: This was fixed in gcloud version 134

Original answer: Run

gcloud auth application-default login --scopes=https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/userinfo.email

Now your remote shell should work again.

Details: I think this was broken by the 128.0.0 update, along with the changes to the gcloud auth login command. The old tokens have the following scopes (according to Google's tokeninfo endpoint):

https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/appengine.admin https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/plus.me

The new tokens from gcloud auth application-default login without any options only have:

https://www.googleapis.com/auth/cloud-platform

This is documented in gcloud auth application-default login --help

Version 134 details: The scopes requested are now:

https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/cloud-platform

See discussion at https://groups.google.com/d/msg/google-appengine/ptc-76K6Kk4/9qr4601BBgAJ