How to read emails with Python and Gmail API

Question

I want to get unread emails from my inbox with python code. I set up google developer account, I made an app (I set it to DESKTOP) and I downloaded credentials.

{"installed":{"client_id":"xxx",
              "project_id":"xxx",
              "auth_uri":"https://accounts.google.com/o/oauth2/auth",
              "token_uri":"https://oauth2.googleapis.com/token",
              "auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs",
              "client_secret":"xxx",
              "redirect_uris":["http://localhost"]
              }
}

This is the code that I have:

import os
from google.oauth2.credentials import Credentials
from googleapiclient.discovery import build

creds = Credentials.from_authorized_user_file(os.path.expanduser('gmail_credencials.json'), ['https://www.googleapis.com/auth/gmail.readonly'])
service = build('gmail', 'v1', credentials=creds)
print(service)
messages = service.users().messages()
print(messages)

But I am getting this error:

ValueError: Authorized user info was not in the expected format, missing fields refresh_token, client_secret, client_id.

I have client_secret and client_id, but I do not have a clue where should I get refresh_token.

Does anyone else has the experience with this error?

Answer 1

The error message you are facing is related to the missing token.json file. This file is created automatically using the credentials.json file (In your case I think it is called gmail_credencials.json based on your code snippet.)

So instead of calling the credentials.json file as your are doing in this part of the code:

creds = Credentials.from_authorized_user_file(
              os.path.expanduser('gmail_credencials.json'), 
              ['https://www.googleapis.com/auth/gmail.readonly']
          )

You need to call the token file that is created after the authorization flow is completed. The authorization flow is this part of the sample below:

 creds = None
    # The file token.json stores the user's access and refresh tokens, and is
    # created automatically when the authorization flow completes for the first
    # time.
    if os.path.exists('token.json'):
        creds = Credentials.from_authorized_user_file('token.json', SCOPES)
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                'credentials.json', SCOPES)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open('token.json', 'w') as token:
            token.write(creds.to_json())

Once the code is created, and stored. You can create the creds (the credentials use for building the services) like this:

SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']

creds = Credentials.from_authorized_user_file('token.json', SCOPES)

The scopes, I add them as variables, so it will be easier to change them if necessary.

Note: You can either rename your credential file to credentials.json or you can replace the sample code with the name of your JSON file.

This is a sample code based on the Google Documentation here. It has an example about the refresh token on it. I'm also listing the files emails that have the label UNREAD. After that, I get the emails with the IDs of the list, and decode the messages payload using base64. Take in consideration that this is just a sample.

The code does the following:

1. Review if the Token is available or does the refresh token process in case the current Token is not valid.
2. List all the emails that are labeled as UNREAD.
3. Using the message ID that we got from the list method and the information of the email.
4. Decode the email information from base64 to ASCII.
5. Get the payload of the message (since the Google messages are multipart).
6. Print the plain text and the html of the email.

from __future__ import print_function

import os.path

from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
import email
import base64 #add Base64
import time 

SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']


def main():
    """Shows basic usage of the Gmail API.
    Lists the user's Gmail labels.
    """
    creds = None
    # The file token.json stores the user's access and refresh tokens, and is
    # created automatically when the authorization flow completes for the first
    # time.
    if os.path.exists('token.json'):
        creds = Credentials.from_authorized_user_file('token.json', SCOPES)
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                'credentials.json', SCOPES)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open('token.json', 'w') as token:
            token.write(creds.to_json())
            
        #Filter and get the IDs of the message I need. 
        #I'm just filtering messages that have the label "UNREAD"

    try:
        service = build('gmail', 'v1', credentials=creds)
        search_id = service.users().messages().list(userId='me', labelIds="UNREAD").execute()
        number_result = search_id['resultSizeEstimate']

        final_list = [] # empty array, all the messages ID will be listed here
        
        # review if the search is empty or not
        # if it has messages on it, It will enter the for

        if number_result>0:
            message_ids = search_id['messages']

            for ids in message_ids:
                final_list.append(ids['id'])
                # call the function that will call the body of the message
                get_message(service, ids['id'] )
                
            return final_list
        
        # If there are not messages with those criterias 
        #The message 'There were 0 results for that search string' will be printed. 

        else:
            print('There were 0 results for that search string')
            return ""

    except HttpError as error:
        # TODO(developer) - Handle errors from gmail API.
        print(f'An error occurred: {error}')
        
        
        #new function to get the body of the message, and decode the message

def get_message(service, msg_id):

    try:
        message_list=service.users().messages().get(userId='me', id=msg_id, format='raw').execute()

        msg_raw = base64.urlsafe_b64decode(message_list['raw'].encode('ASCII'))

        msg_str = email.message_from_bytes(msg_raw)

        content_types = msg_str.get_content_maintype()
        
        #how it will work when is a multipart or plain text

        if content_types == 'multipart':
            part1, part2 = msg_str.get_payload()
            print("This is the message body, html:")
            print(part1.get_payload())
            return part1.get_payload()
        else:
            print("This is the message body plain text:")
            print(msg_str.get_payload())
            return msg_str.get_payload()


    except HttpError as error:
        # TODO(developer) - Handle errors from gmail API.
        print(f'An error occurred: {error}')


if __name__ == '__main__':
    main()

---

Reference:

• Method: users.messages.list
• Method: users.messages.get
• Format
• Google Documentation "Python quickstart"
• base64 Python
• Get an OAuth2 Refresh Token and Configure Your Client