using AWS SNS and Lambda - what's the right use case for an activity feed

Question

I want to use an AWS lambda function to fan out and insert activity stream info to a firebase endpoint for every user.

Should I be using Kinesis, SQS or SNS to trigger the lambda function for this use case? The updates to the activity stream can be triggered from the server and clients should receive the update near real time (within 60 seconds or so).

I think I have a pretty good idea on what SQS is, and have used Kinesis in the past but not quite sure about SNS.

If we created an SNS topic for each user and then each follower subscribes to these topics with an AWS lambda function - would that work?

Does it make sense to programmatically create topics and subscriptions for every user and follow relationship respectively?

Answer 1

As usual, answer to such a question is mostly, 'it depends on your use-case'.

Kinesis vs SQS:

If your clients care about relative (timestamp-based, for e.g.) ordering between events, you'll almost certainly have to go with Kinesis. SQS is a best-effort FIFO queue, meaning events can arrive out of order and it would up to your client to manage relative ordering.

As far as latencies are concerned, I have seen that data ingested into Kinesis can become visible to its consumer in as less as 300 ms.

When can SNS be interesting to you?

(Even with SNS, you'd have to use SQS). If you use SNS, it will be easy to add a new application that can process your events. For example, if in future you decide to ingest all events into, say, an Elasticsearch to provide real-time analytics, all you'd have to do is add another SQS queue to your existing topic(s) and write a consumer.