How to avoid 'Failed to retrieve access token: { "error" : "invalid_grant" }' in offline GAE cron tasks?

Question

This post is a followup to How to make 'access_type=offline' / server-only OAuth2 operations on GAE/Python. The http = credentials.authorize(httplib2.Http()) part no longer fails when testing, but it seems it still does when run by GAE's cron, where it's unable to refresh my access_token :

1. I can manually run my job by calling /fetch, say at 11:45.
2. Scheduling immediately a /cronfetch job at 11:55 works then, without any access_token issue.

3. But then, I woke up this morning seeing that the same /cronfetch task (same except the timing, which is at 01:00 for my non-test daily task) failed:

   I 2013-06-10 05:53:51.324 make: Got type <class 'google.appengine.api.datastore_types.Blob'>
   I 2013-06-10 05:53:51.325 validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
   I 2013-06-10 05:53:51.327 URL being requested: https://www.googleapis.com/youtube/v3/playlists?alt=json&part=snippet%2Cstatus
   I 2013-06-10 05:53:51.397 Refreshing due to a 401
   I 2013-06-10 05:53:51.420 make: Got type <class 'google.appengine.api.datastore_types.Blob'>
   I 2013-06-10 05:53:51.421 validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
   I 2013-06-10 05:53:51.421 Refreshing access_token
   I 2013-06-10 05:53:51.458 Failed to retrieve access token: { "error" : "invalid_grant" }
   I 2013-06-10 05:53:51.468 make: Got type <class 'google.appengine.api.datastore_types.Blob'>
   I 2013-06-10 05:53:51.468 validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
   I 2013-06-10 05:53:51.471 validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
   I 2013-06-10 05:53:51.471 get: Got type <class 'oauth2client.appengine.CredentialsModel'>
   E 2013-06-10 05:53:51.480 invalid_grant Traceback (most recent call last): File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in
   

This Getting "invalid_grant" error on token refresh mailing list message (+ SO post 1, SO post 2, SO post 3) look similar to my problem, but it seems to be happening with a access_type=online token. In my case I just use the default access_type=offline, and I see the "Perform these operations when I'm not using the application" mention in the initial access request.

I just re-scheduled a cron run at 08:25 (taking care not to launch a manual one) with debug print statements that I committed to GitHub for you. Here is what I get, it is similar but not identical (Note the few last lines seem ordered incorrectly, I definitely am not doing OAuth2 stuff in create_playlist until all sources are read). So ignoring the skewed order (GAE logging artifact?), it seems my http = credentials.authorize(Http()) call in create_playlist(self), currently at line 144 is wrong:

    ...
    E 2013-06-10 08:26:12.817 http://www.onedayonemusic.com/page/2/ : found embeds ['80wWl_s-HuQ', 'kb1Nu75l1vA', 'kb1Nu75l1vA', 'RTWcNRQtkwE', 'RTWcNRQtkwE', 'ZtDXezAhes8', 'ZtDXezAhes8', 'cFGxNJhKK9c', 'cFGxNJhKK9c'
    I 2013-06-10 08:26:14.019 make: Got type <class 'google.appengine.api.datastore_types.Blob'>
    I 2013-06-10 08:26:14.020 validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
    I 2013-06-10 08:26:14.022 URL being requested: https://www.googleapis.com/youtube/v3/playlists?alt=json&part=snippet%2Cstatus
    I 2013-06-10 08:26:14.100 Refreshing due to a 401
    I 2013-06-10 08:26:14.105 make: Got type <class 'google.appengine.api.datastore_types.Blob'>
    I 2013-06-10 08:26:14.106 validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
    I 2013-06-10 08:26:14.106 Refreshing access_token
    E 2013-06-10 08:26:18.994 Deadline exceeded while waiting for HTTP response from URL: https://accounts.google.com/o/oauth2/token Traceback (most recent call last): File "/pyt
    E 2013-06-10 08:26:18.996 http://www.onedayonemusic.com/page/3/ : found embeds ['80wWl_s-HuQ', '6VNu2MLdE0c', '6VNu2MLdE0c', 'YwQilKbK9Mk', 'YwQilKbK9Mk', 'KYdB3rectmc', 'KYdB3
    E 2013-06-10 08:26:18.996 crawl_videos end
    E 2013-06-10 08:26:18.996 create_playlist start
    E 2013-06-10 08:26:18.996 create_playlist got creds
    E 2013-06-10 08:26:18.996 create_playlist authorized creds

→ Why does the cron job work 5min after a manual run, but fails 6hours later? I thought the refresh token never expired. What am I doing wrong?

Note that's my first GAE work, and my second Python program at all, general code review/advice is very welcome but please be gentle :)

The code is on GitHub and my instance can be reached at dailygrooves.org. Thanks for your help!

Answer 1

An invalid_grant is returned when the refresh token can't be used to get a new access token from the current user. This is happening to you because the stored Credentials object has a null refresh token, i.e.

>>> credentials.refresh_token is None
True

As mentioned in the NOTE in How to make 'access_type=offline' / server-only OAuth2 operations on GAE/Python?:

If a user has already authorized your client ID, the subsequent times you perform OAuth for these users they will not see the OAuth dialog and you won't be given a refresh token.

You need to make sure your Credentials are stored with a valid refresh token and the easiest way to do this, as mentioned in your last question as well as in all 3 questions you linked to is to use approval_prompt=force when creating your OAuth2WebServerFlow or OAuth2Decorator object (whichever you are using).