Google Pubsub Subscription based on attributes or Message content

Question

I am relatively new to GCP platform. I have a system which is publishing messages in pubsub topic. Messages has an attribute named country and based on this attribute different clients would like to subscribe to messages. Client X is interested only in messages for country a. Client Y is interested only in messages of country b.

Will I have to create topic for every country in google pub sub? or there is a smart way to have subscription on single topic based on attribute value

I am referring to attribute passed by publisher as stated in google docs. https://cloud.google.com/pubsub/docs/publisher

Thanks

Answer 1

Update June 2020: Filtering is now an available feature in Google Cloud Pub/Sub. When creating a subscription, one can specify a filter that looks at message attributes. If a message does not match the filter, the Pub/Sub service automatically acknowledges the message without delivering it to the subscriber.

In this specific case, you can create a single topic and two different subscriptions, one that is meant to get messages for country a and one that is meant to get messages for country b. The filters for each subscription would be:

attributes.country = a

attributes.country = b

Previous answer:

The feature you are talking about is called filtering: you want the ability for a subscription to specify that it wants to receive a subset of the messages based on the attributes provided in the message. At this time, that feature does not exist in Google Cloud Pub/Sub.

There are two ways to handle this right now:

1. Filter the messages in the subscribers themselves by looking at the attributes and immediately acking all messages that they are not interested in. This does mean you will pay for delivery of all of the messages to each subscriber which may not be desirable depending on the percentage of messages the subscriber is actually interested in.
2. Create separate topics and a subscription on each topic, publish messages to those individual topics based on the attributes, and then have subscribers get messages on the subscription for the appropriate topic.

We are exploring ways to add functionality that will make this use case easier in the future.