Google.GData.Client.GDataRequestException - Authentication suddenly fails in old code

Question

I'm suddenly starting to get the following exception when attempting to authenticate and access a spreadsheet on Google drive:

Unhandled Exception: Google.GData.Client.GDataRequestException: Execution of aut hentication request returned unexpected result: 404 at Google.GData.Client.Utilities.getAuthException(TokenCollection tokens, Htt pWebResponse response) at Google.GData.Client.Utilities.QueryClientLoginToken(GDataCredentials gc, S tring serviceName, String applicationName, Boolean fUseKeepAlive, IWebProxy prox yServer, Uri clientLoginHandler) at Google.GData.Client.GDataGAuthRequest.QueryAuthToken(GDataCredentials gc) at Google.GData.Client.GDataGAuthRequest.EnsureCredentials() at Google.GData.Client.GDataRequest.EnsureWebRequest() at Google.GData.Client.GDataGAuthRequest.EnsureWebRequest() at Google.GData.Client.GDataRequest.Execute() at Google.GData.Client.GDataGAuthRequest.Execute(Int32 retryCounter) at Google.GData.Client.GDataGAuthRequest.Execute() at Google.GData.Client.Service.Query(Uri queryUri, DateTime ifModifiedSince, String etag, Int64& contentLength) at Google.GData.Client.Service.Query(FeedQuery feedQuery) at Google.GData.Documents.DocumentsService.Query(DocumentsListQuery feedQuery ) at GoogleLogger.GoogleService.getLastXECLogRows(String folderName, String fileName, Int32 rows)

This is in code that has been running for two years without any problems. I first thought that I may have lost access permissions on my production system but Google drive loads fine in my web browser. Tried it on several other systems and am getting the very same.

Did they change something in the Google API today? This can't be coincidence!

Answer 1

Google has retired their older authentication API. OAuth 2.0 should be used instead.

I spent too much time to figure out how to use newer Auth API with older GData API grabbing bits and pieces of information here and there from the Internet. I decided to share all the the details with screenshots to save your time.

1. Go to https://console.developers.google.com/project

2. Hit Create Project button

3. Create project. Type in some name.

4. Go to API & Auth > Credentials and hit Create new Client ID button. It will create JSON key for you automatically - ignore that.

5. Hit Generate new P12 key

6. File download will start automatically. Remember the password, you will need it to open the file you just downloaded.

7. Rename downloaded file to Key.p12 and add it to your solution. Make sure you set Build Action and Copy to Output Directory accordingly.

8. Install Google API Auth using Nuget. Type the following in the Package Manager Console

   Install-Package Google.Apis.Auth
   

9. Copy service account email address that was generated in Step #4.

10. Grant appropriate permission to this user in your Google Spreadsheet.

11. Use the following code to query the spreadsheet. Replace email and Google spreadsheet URL address in the code below.

    const string ServiceAccountEmail = "452351479-q41ce1720qd9l94s8847mhc0toao1fed@developer.gserviceaccount.com";
    
    var certificate = new X509Certificate2("Key.p12", "notasecret", X509KeyStorageFlags.Exportable);
    
    var serviceAccountCredentialInitializer = 
        new ServiceAccountCredential.Initializer(ServiceAccountEmail)
        {
            Scopes = new[] { "https://spreadsheets.google.com/feeds" }
        }.FromCertificate(certificate);
    
    var credential = new ServiceAccountCredential(serviceAccountCredentialInitializer);
    
    if (!credential.RequestAccessTokenAsync(System.Threading.CancellationToken.None).Result)
        throw new InvalidOperationException("Access token request failed.");
    
    var requestFactory = new GDataRequestFactory(null);
    requestFactory.CustomHeaders.Add("Authorization: Bearer " + credential.Token.AccessToken);
    
    var service = new SpreadsheetsService(null) { RequestFactory = requestFactory };
    
    var query = new ListQuery("https://spreadsheets.google.com/feeds/list/0ApZkobM61WIrdGRYshh345523VNsLWc/1/private/full");
    var feed = service.Query(query);
    
    var rows = feed.Entries
        .Cast<ListEntry>()
        .Select(arg =>
            new
            {
                Field0 = arg.Elements[0].Value,
                Field1 = arg.Elements[1].Value
            })
        .ToList();
    

Answer 2

I've managed to solve this by using this solution with Service Account with oAuth2.0 Accessing older GData APIs (Spreadsheet API) using OAuth 2 and a service account

The solution: 1. Create Project and Google Service Account in https://console.developers.google.com/project

2. Generate your p12 key.
3. Allow APIs in Developer console you want to use (basically we are going to use old API, so you can skip this step, but just in case)
4. Use the code below (.NET Framework 4.5!)
5. Also don't forget to grant "[email protected]" access to your spreadsheet document as you grant permissions for usual users by pressing Share.

Code:

using System.Security.Cryptography.X509Certificates;
using Google.GData.Client;
using Google.GData.Extensions;
using Google.GData.Spreadsheets;
using Google.Apis.Auth.OAuth2;

string keyFilePath = @"C:\key.p12";    // found in developer console
string serviceAccountEmail = "[email protected]";   // found in developer console
var certificate = new X509Certificate2(keyFilePath, "notasecret", X509KeyStorageFlags.Exportable);

ServiceAccountCredential credential = new ServiceAccountCredential(new ServiceAccountCredential.Initializer(serviceAccountEmail) //create credential using certificate
        {
            Scopes = new[] { "https://spreadsheets.google.com/feeds/" } //this scopr is for spreadsheets, check google scope FAQ for others
        }.FromCertificate(certificate));

credential.RequestAccessTokenAsync(System.Threading.CancellationToken.None).Wait(); //request token

var requestFactory = new GDataRequestFactory("Some Name"); 
requestFactory.CustomHeaders.Add(string.Format("Authorization: Bearer {0}", credential.Token.AccessToken));

SpreadsheetsService myService = new SpreadsheetsService("You App Name"); //create your old service
myService.RequestFactory = requestFactory; //add new request factory to your old service

SpreadsheetQuery query = new SpreadsheetQuery(); //do the job as you done it before
SpreadsheetFeed feed = myService.Query(query);

Answer 3

Alright, I figured it out. Step by step instructions as follows - also see the code I provided below. FYI, this runs in .Net 3.5 and unlike the solution offered previously there are no new dependencies. You should be up and running in no time.

1. If you haven't yet create your OAuth 2.0 credentials - I assume you already know how to get those but here:

   a) Log into your Google developer console: http://console.developers.google.com b) Create a project c) Create your credentials - use 'installed application' d) add the APIs that you need - I think Drive API is definitely required. I also added Drive SDK just in case.

2. Copy the code below into VS and edit the first Main() method with your client key and secret key.

3. Run the app and copy both the new access token and the refresh token. Put those and your remaining credentials into the second Main() method below.

4. You now should be able to run the second Main() method (just reverse the naming). From now on that will be all you need - there is no need to re-run the first Main() method.

BTW, the first Main() method below was found here: https://developers.google.com/google-apps/spreadsheets/authorize

I did however add the missing token type as well the access type. Those are needed, so use the code below:

using System;
using Google.GData.Client;
using Google.GData.Spreadsheets;
using Google.GData.Documents;
using System.Configuration;
using System.Collections.Specialized;

namespace GoogleSpreadsheet
{
  class GoogleOAutho2
  {

    private static String folderName = "crazy.ivan";


    static void Main(string[] args)
    {
      ////////////////////////////////////////////////////////////////////////////
      // STEP 1: Configure how to perform OAuth 2.0
      ////////////////////////////////////////////////////////////////////////////

      // TODO: Update the following information with that obtained from
      // https://code.google.com/apis/console. After registering
      // your application, these will be provided for you.

      string CLIENT_ID = "your_client_id";

      // This is the OAuth 2.0 Client Secret retrieved
      // above.  Be sure to store this value securely.  Leaking this
      // value would enable others to act on behalf of your application!
      string CLIENT_SECRET = "your_secret_key"

      // Space separated list of scopes for which to request access.
      string SCOPE = "https://www.googleapis.com/auth/drive https://spreadsheets.google.com/feeds https://docs.google.com/feeds";

      // This is the Redirect URI for installed applications.
      // If you are building a web application, you have to set your
      // Redirect URI at https://code.google.com/apis/console.
      string REDIRECT_URI = "urn:ietf:wg:oauth:2.0:oob";

      string TOKEN_TYPE = "refresh";

      ////////////////////////////////////////////////////////////////////////////
      // STEP 2: Set up the OAuth 2.0 object
      ////////////////////////////////////////////////////////////////////////////

      // OAuth2Parameters holds all the parameters related to OAuth 2.0.
      OAuth2Parameters parameters = new OAuth2Parameters();

      // Set your OAuth 2.0 Client Id (which you can register at
      // https://code.google.com/apis/console).
      parameters.ClientId = CLIENT_ID;

      // Set your OAuth 2.0 Client Secret, which can be obtained at
      // https://code.google.com/apis/console.
      parameters.ClientSecret = CLIENT_SECRET;

      // Set your Redirect URI, which can be registered at
      // https://code.google.com/apis/console.
      parameters.RedirectUri = REDIRECT_URI;

      ////////////////////////////////////////////////////////////////////////////
      // STEP 3: Get the Authorization URL
      ////////////////////////////////////////////////////////////////////////////

      // Set the scope for this particular service.
      parameters.Scope = SCOPE;

      parameters.AccessType = "offline"; // IMPORTANT and was missing in the original

      parameters.TokenType = TOKEN_TYPE; // IMPORTANT and was missing in the original


      // Get the authorization url.  The user of your application must visit
      // this url in order to authorize with Google.  If you are building a
      // browser-based application, you can redirect the user to the authorization
      // url.
      string authorizationUrl = OAuthUtil.CreateOAuth2AuthorizationUrl(parameters);
      Console.WriteLine(authorizationUrl);
      Console.WriteLine("Please visit the URL above to authorize your OAuth "
        + "request token.  Once that is complete, type in your access code to "
        + "continue...");
      parameters.AccessCode = Console.ReadLine();

      ////////////////////////////////////////////////////////////////////////////
      // STEP 4: Get the Access Token
      ////////////////////////////////////////////////////////////////////////////

      // Once the user authorizes with Google, the request token can be exchanged
      // for a long-lived access token.  If you are building a browser-based
      // application, you should parse the incoming request token from the url and
      // set it in OAuthParameters before calling GetAccessToken().
      OAuthUtil.GetAccessToken(parameters);
      string accessToken = parameters.AccessToken;
      string refreshToken = parameters.RefreshToken;
      Console.WriteLine("OAuth Access Token: " + accessToken + "\n");
      Console.WriteLine("OAuth Refresh Token: " + refreshToken + "\n");

      ////////////////////////////////////////////////////////////////////////////
      // STEP 5: Make an OAuth authorized request to Google
      ////////////////////////////////////////////////////////////////////////////

      // Initialize the variables needed to make the request
      GOAuth2RequestFactory requestFactory =
          new GOAuth2RequestFactory(null, "MySpreadsheetIntegration-v1", parameters);
      SpreadsheetsService service = new SpreadsheetsService("MySpreadsheetIntegration-v1");
      service.RequestFactory = requestFactory;

      // Make the request to Google
      // See other portions of this guide for code to put here...

      // Instantiate a SpreadsheetQuery object to retrieve spreadsheets.
      Google.GData.Spreadsheets.SpreadsheetQuery query = new Google.GData.Spreadsheets.SpreadsheetQuery();

      // Make a request to the API and get all spreadsheets.
      SpreadsheetFeed feed = service.Query(query);

      // Iterate through all of the spreadsheets returned
      foreach (SpreadsheetEntry entry in feed.Entries)
      {
        // Print the title of this spreadsheet to the screen
        Console.WriteLine(entry.Title.Text);
      }
      Console.ReadLine();

     }


    // once you copied your access and refresh tokens
    // then you can run this method directly from now on...
    static void MainX(string[] args)
    {
    GOAuth2RequestFactory requestFactory = RefreshAuthenticate();

         SpreadsheetsService service = new SpreadsheetsService("MySpreadsheetIntegration-v1");  
         service.RequestFactory = requestFactory;

      // Instantiate a SpreadsheetQuery object to retrieve spreadsheets.
      Google.GData.Spreadsheets.SpreadsheetQuery query = new Google.GData.Spreadsheets.SpreadsheetQuery();

      // Make a request to the API and get all spreadsheets.
      SpreadsheetFeed feed = service.Query(query);

      // Iterate through all of the spreadsheets returned
      foreach (SpreadsheetEntry entry in feed.Entries)
      {
        // Print the title of this spreadsheet to the screen
        Console.WriteLine(entry.Title.Text);
      }
      Console.ReadLine();



    public static GOAuth2RequestFactory RefreshAuthenticate() {
        OAuth2Parameters parameters = new OAuth2Parameters(){
            RefreshToken = "the_refresh_token_you_copied_from_the_CLI_running_the_first_method";
            AccessToken = "the_access_token_you_copied_from_the_CLI_running_the_first_method";
            ClientId = "your_client_id";
            ClientSecret = "your_dirty_little_secret";
            Scope = "https://www.googleapis.com/auth/drive https://spreadsheets.google.com/feeds",
            AccessType = "offline",
            TokenType = "refresh"
        };
        string authUrl = OAuthUtil.CreateOAuth2AuthorizationUrl(parameters);
        return new GOAuth2RequestFactory(null, "evilspeculator", parameters);
    }
  }
}

Hope that works for you guys - best of luck!