What is the correct mechanism to authenticate to Google in a standalone Python script?

Question

I've got some code that I'm using to extract email address from Gmail contacts into text file. It's a simple Python script that runs in a cron job, and is based on the Python gdata library (currently v2.0.18).

As of earlier this month, this no longer works due to Google deprecating the ClientLogin protocol. The resulting error looks like this:

{'status': 401, 'body': '<?xml version="1.0" encoding="UTF-8"?>\n<errors xmlns="http://schemas.google.com/g/2005">\n <error>\n  <domain>GData</domain>\n  <code>required</code>\n  <location type="header">Authorization</location>\n  <internalReason>Login Required</internalReason>\n </error>\n</errors>\n', 'reason': 'Unauthorized'}

I knew this was coming and dealt with it in other places (like AppEngine applications), but forgot that I would have to convert this script. Now that I'm in here, I find that I have no idea how I'm supposed to make this work.

All of the references I've found, such as here on the Google Apps Developer Blog or here and here on StackOverflow, suggest that the solution is to use an OAuth2Token. However, that requires a client id and client secret from the Google APIs console -- which is tied to an application. I don't have an application. I just want to authenticate from my script.

Can someone please suggest the proper way to do this in a standalone script? Or am I out of luck and there's no mechanism to accomplish this any more?

This is the guts of the existing contacts code:

from gdata.contacts.service import ContactsService, ContactsQuery

user = "[email protected]"
password = "mypassword"

addresses = set()
client = ContactsService(additional_headers={"GData-Version":"2"})
client.ssl = True
client.ClientLogin(user, password)
groups = client.GetGroupsFeed()
for group in groups.entry:
   if group.content.text == "System Group: My Contacts":
      query = ContactsQuery()
      query.max_results = 9999   # large enough that we'll get "everything"
      query.group = group.id.text
      contacts = client.GetContactsFeed(query.ToUri())
      for contact in contacts.entry:
         for email in contact.email:
            addresses.add(email.address.lower())
      break
return addresses

Ideally, I'm looking to replace client.ClientLogin() with some other mechanism that preserves the rest of code using gdata. Alternately, if this can't really be done with gdata, I'm open to converting to some other library that offers similar functionality.

Answer 1

It ended up being easier to just hack together a shell script using curl than mess with the gdata library. As expected, I was able to do most of the verification process manually, outside of the script, per the OAuth2 Device Flow instructions.

After finishing the verification process, I had the 4 required credentials: the client id, the client secret, the access token, and the refresh token. Per Google's documentation, the access token eventually expires. You can get a new access token by asking the token manager to refresh the token. When you do this, you apparently get a new access token, but not a new refresh token.

I store the client id and secret and the refresh token in the CREDENTIALS file in JSON format. Since the access token changes over time, it is stored in the ACCESS file, also in JSON format.

The important parts of the script are shown below:

#!/bin/ksh

CLIENT_ID=$(cat ${CREDENTIALS} | jq -r ".client_id")
CLIENT_SECRET=$(cat ${CREDENTIALS} | jq -r ".client_secret")
REFRESH_TOKEN=$(cat ${CREDENTIALS} | jq -r ".refresh_token")
ACCESS_TOKEN=$(cat ${ACCESS} | jq -r ".access_token")

CONTACTS_URL="https://www.google.com/m8/feeds/contacts/default/full?access_token=${ACCESS_TOKEN}&max-results=5000&v=3.0"
ERROR=$(curl --show-error --silent --fail "${CONTACTS_URL}" -o ${CONTACTS_XML} 2>&1)
RESULT=$?
if [[ ${RESULT} -eq 0 ]]
then
   cat ${CONTACTS_XML} | grep 'gd:email' | sed 's/^.*address="//g' | sed 's/".*$//g' | tr '[:upper:]' '[:lower:]' | sort | uniq
elif [[ ${RESULT} -eq 22 ]]
then
   echo "${ERROR}" | grep -q "401"
   if [[ $? -eq 0 ]]
   then
      TOKEN_URL="https://www.googleapis.com/oauth2/v3/token"
      REFRESH_PARAMS="client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&refresh_token=${REFRESH_TOKEN}&grant_type=refresh_token"
      ERROR=$(curl --show-error --silent --fail --data "${REFRESH_PARAMS}" ${TOKEN_URL} -o ${REFRESH_JSON})
      RESULT=$?
      if [[ ${RESULT} -eq 0 ]]
      then
         ACCESS_TOKEN=$(cat ${REFRESH_JSON} | jq -r ".access_token")
         jq -n --arg access_token "${ACCESS_TOKEN}" '{"access_token": $access_token, }' > ${ACCESS}

         CONTACTS_URL="https://www.google.com/m8/feeds/contacts/default/full?access_token=${ACCESS_TOKEN}&max-results=5000&v=3.0"
         ERROR=$(curl --show-error --silent --fail "${CONTACTS_URL}" -o ${CONTACTS_XML} 2>&1)
         RESULT=$?
         if [[ ${RESULT} -eq 0 ]]
         then
            cat ${CONTACTS_XML} | grep 'gd:email' | sed 's/^.*address="//g' | sed 's/".*$//g' | tr '[:upper:]' '[:lower:]' | sort | uniq
         else
            print "Unexpected error: ${ERROR}" >&2
            exit 1
         fi
      else
         print "Unexpected error: ${ERROR}" >&2
         exit 1
      fi
   else
      print "Unexpected error: ${ERROR}" >&2
      exit 1
   fi
else
   print "Unexpected error: ${ERROR}" >&2
   exit 1
fi

It's not the prettiest thing in the world, but I was looking for something quick-and-dirty, and this works.