Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unable to retrieve message content after Google Pub/Sub webhook call to server

I've subscribed to Google Cloud Pub/Sub, created a topic and created a subscription with a webhook that points to a URL at my server. I've then used the Gmail API to call watch on a particular label that I've created in my Gmail account and associated it with the topic I created.

When I messages arrive into the label in my Gmail account, the webhook is triggered a POST request to the URL at my server. But no matter what the content of the email is, the POST body is always something like this:

{
    "message": {
        "attributes": {},
        "data": "eyJlbWFpbEFkZHJlc3MiOiJteWVtYWlsQG15ZG9tYWluLmNvbSIsImhpc3RvcnlJZCI6MTIzNDU2N30K",
        "messageId": "12345678900000", # only relevant to Pub/Sub
        "message_id": "12345678900000",
        "publishTime": "2017-04-16T15:42:08.968Z",
        "publish_time": "2017-04-16T15:42:08.968Z"
    },
    "subscription": "projects/proj-name/subscriptions/sub-name"
} 

The data field is Base64 encoded, which in this case is:

{"emailAddress":"[email protected]","historyId":1234567}

That's what I get, every time, no matter what the email content is. However, if run a publish test from the google developer console, the base64 encoded value in the data field is the actual message string that I specified.

I've tried making a subsequent call to history.list with the Gmail API using the historyId from the base64 decoded data value, but all I get back is a response like this:

{"historyId": "1234567"}

How am I suppose to get the email content?

like image 578
RTF Avatar asked Apr 16 '17 16:04

RTF


People also ask

Where are Pub/Sub messages stored?

Cloud Pub/Sub uses its own underlying storage system to store messages and does not store them in Cloud Storage. There is no way to access the underlying files in which messages are stored.

How do I read a Pub/Sub message?

In the console, go to the Pub/Sub subscriptions page. Click the subscription ID. Click the Messages tab. Click Pull.

How does Pub/Sub deliver messages to endpoints?

The Pub/Sub server sends each message as an HTTPS request to the subscriber client at a pre-configured endpoint. This request is shown as a PushRequest in the image. The endpoint acknowledges the message by returning an HTTP success status code. A non-success response indicates that Pub/Sub must resend the messages.

How long do messages stay in Pubsub?

The default message retention duration is 7 days and the default expiration period is 31 days. To create a subscription with retention of acked messages enabled, follow these steps: In the console, go to the Pub/Sub subscriptions page.


1 Answers

A bit late but others might be interested. The answer is here. Basically you get an historyId from the watch call. Store it. Then when you get a notification from Gmail:

  • keep track of the historyId
  • fetch the history list (gmail.users.history.list) using the stored historyId (from the watch call)
  • import all the desired messages from the history
  • once you're done, replace the current historyId by the one from the notification.

Hope it helps.

like image 62
jboga Avatar answered Nov 15 '22 10:11

jboga