Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Google pubsub into HTTP triggered cloud function?

Is it possible to trigger an HTTP cloud function in response to a pubsub message?

When editing a subscription, google makes it possible to push the message to an HTTPS endpoint, but for abuse reasons one has to be able to prove that you own the domain in order to do this, and of course you can't prove that you own google's own *.cloudfunctions.net domain which is where they get deployed.

The particular topic I'm trying to subscribe to is a public one, projects/pubsub-public-data/topics/taxirides-realtime. The answer might be use a background function rather than HTTP triggered, but that doesn't work for different reasons:

gcloud functions deploy echo --trigger-resource projects/pubsub-public-data/topics/taxirides-realtime --trigger-event google.pubsub.topic.publish ERROR: gcloud crashed (ArgumentTypeError): Invalid value 'projects/pubsub-public-data/topics/taxirides-realtime': Topic must contain only Latin letters (lower- or upper-case), digits and the characters - + . _ ~ %. It must start with a letter and be from 3 to 255 characters long.

This seems to indicate this is only permitted on topics I own, which is a strange limitation.

like image 504
FrobberOfBits Avatar asked Oct 16 '22 15:10

FrobberOfBits


1 Answers

It is possible to publish from a pub/sub topic to a cloud function. I was looking for a way to publish messages from a topic in project A to a function in project B. This was not possible with a regular topic trigger, but it is possible with http-trigger. Overall steps to follow:

  • Creata a http-triggered function in project B.
  • Create a topic in project A.
  • Create a push subscription on that topic in project A.
  • Domain verification

Push subscription

enter image description here

Here we have to fill in three things: the endpoint, the audience and the service account under which the function runs.

  • Push Endpoint: https://REGION-PROJECT_ID.cloudfunctions.net/FUNC_NAME/ (slash at end)
  • Audience: https://REGION-PROJECT_ID.cloudfunctions.net/FUNC_NAME (no slash at end)
  • Service Account: Choose a service account under which you want to send the actual message. Be sure the service account has the "roles/cloudfunctions.invoker" role on the cloud function that you are sending the messages to. Since november 2019, http-triggered functions are automatically secured because AllUsers is not set by default. Do not set this property unless you want your http function to be public!

Domain verification

Now you probably can't save your subscription because of an error, that is because the endpoint is not validated by Google. Therefore you need to whitelist the function URL at: https://console.cloud.google.com/apis/credentials/domainverification?project=PROJECT_NAME.

Following this step will also bring you to the Google Search Console, where you would also need to verify you own the endpoint. Sadly, at the time of writing this process cannot be automated.

  • Next we need to add something in the lines of the following (python example) to your cloud function to allow google to verify the function:

    if request.method == 'GET':
        return '''
            <html>
                <head>
                    <meta name="google-site-verification" content="{token}" />
                </head>
                <body>
                </body>
            </html>
        '''.format(token=config.SITE_VERIFICATION_CODE)
    

Et voila! This should be working now.

Sources: https://github.com/googleapis/nodejs-pubsub/issues/118#issuecomment-379823198, https://cloud.google.com/functions/docs/calling/http

like image 119
Nebulastic Avatar answered Oct 21 '22 08:10

Nebulastic