Gmail watch user inbox, history.getMessagesAdded is not returning the new message

Question

Requirement is to sync mails from Gmail for an user into our CRM. The system in place is based on Google Pub/Sub which watches inbox of the user for any change and fires the notification to our HTTPs endpoint. More on this at Gmail cloud pub/sub.

Based on the above procedure we git history of changes. And then i am interested in only new messages, so history.getMessagesAdded is preferred as per this guide. Issue we are facing now is the first mail of a thread is not captured under messagesAdded all the subsequent messages are passing through our system.

Note: For the first mail, we do get push from Google. But when we try to get Messages added it turns out empty. Is there anything special needs to be done for the first mail of the thread or am i missing out something.

Answer 1

I was experiencing a very similar problem, and my mistake was that I was using the historyId from the push notification, the solution was to store the last known historyId on my database, so, every time I get a notification, I get the history from the id I have stored, not the one from the notification.

In my case, the historyId from the notification doesn't even make part of the history, maybe because of my watch restrictions: labelIds=['INBOX']

This is the google pub/sub notification:

{
  message:
  {
    data: {"emailAddress": "[email protected]", "historyId": "9876543210"},
    message_id: "1234567890",
  }

  subscription: "projects/myproject/subscriptions/mysubscription"
}

I was using the message.data.historyId, wich was causing the confusion!

The message.data, comes as a base64 encoded string, in this example I just decoded it!

Step by step for watching new e-mails on the inbox:

1. Do all the configuration in the google pub/sub.

2. Start watching the user with the filters you want (docs.: https://developers.google.com/gmail/api/v1/reference/users/watch)

3. Store the historyId obtained in the step 2

4. When receive the notification, get all the events (history) using the stored id as the startHistoryId parameter (docs: https://developers.google.com/gmail/api/v1/reference/users/history/list)

5. In the history list obtained on the step 4, look for the new messages: history.getMessagesAdded().

6. Update the last known history id in your database, so you don't need to deal with the whole history every time!

I hope it helps.