how to use gpg encrypted oauth files via Python for offlineimap

Question

I was playing around with oauth2 to get a better understanding of it. For this reason, I've installed offlineimap which should act as a third-party app. I've found a nice way to read encrypted credentials here on stackexchange.

Based on the linked post I've modified/copied the following python script:

import subprocess
import os
import json

def passwd(file_name):
  acct = os.path.basename(file_name)
  path = "/PATHTOFILE/%s" % file_name
  args = ["gpg", "--use-agent", "--quiet", "--batch", "-d", path]
  try:
    return subprocess.check_output(args).strip()
  except subprocess.CalledProcessError:
    return ""

def oauthpasswd(acct, key):
  acct = os.path.basename(acct)
  path = "/PATHTOFILE/%s_oauth2.gpg" % acct
  args = ["gpg", "--use-agent", "--quiet", "--batch", "-d", path]
  try:
    return str(json.loads(subprocess.check_output(args).strip())['installed'][key])
  except subprocess.CalledProcessError:
    return ""

def prime_gpg_agent():
  ret = False
  i = 1
  while not ret:
    ret = (passwd("prime.gpg") == "prime")
    if i > 2:
      from offlineimap.ui import getglobalui
      sys.stderr.write("Error reading in passwords. Terminating.\n")
      getglobalui().terminate()
    i += 1
  return ret

prime_gpg_agent()

In the corresponding offlineimaprc file I call the function with the correct arguments:

oauth2_client_id = oauthpasswd('gmail', 'client_id')
oauth2_client_secret = oauthpasswd('gmail', 'client_secret')
oauth2_request_url = https://accounts.google.com/o/oauth2/token
oauth2_refresh_token = passwd('gmail_rf_token.gpg')

Please note in the local file the PATHTOFILE is set correctly. What I've done was downloaded the JSON file from Google including the oauth2 credentials and encrypted it. I've stored the refresh token in a separate file. However, if I run offlineimap I get an authentication error:

 ERROR: While attempting to sync account 'gmail'
  ('http error', 401, 'Unauthorized', <httplib.HTTPMessage instance at 0x7f488c214320>) (configuration is: {'client_secret': "oauthpasswd('gmail', 'client_secret')", 'grant_type': 'refresh_token', 'refresh_token': "passwd('gmail_rf_token.gpg')", 'client_id': "oauthpasswd('gmail', 'client_id')"})

I've tried then to check the outputs of the two python functions passwd and oauthpasswd in a python interpreter. I get the desired outputs. Even more, I've copied the output from the functions within the python interpreter to the offlineimaprc config file and I was able to sync to Gmail. This implies that there must be a mistake when offlineimap executes the file but I can't see what's wrong.

If I only encrypt my Gmail password everything is working. This means there is something going wrong from the details downloaded from Google (client_id, client_secret and refresh token). As pointed out above, the values itself are correct. I've really copied the output of

oauthpasswd('gmail', 'client_id')
oauthpasswd('gmail', 'client_secret')
passwd('gmail_rf_token.gpg')

from a python console to the offlineimaprc file and it worked.

Answer 1

The problem which happens is the following. According to this answer offlineimap does not allow for encryption of all keys within the offlinemaprc file. That's why the python function never gets evaluated and the wrong strings are handed over.