Can I persist SNS (or SQS) messages to S3 through given AWS integrations?

Question

I want to save all SNS messages to S3 so I can keep a complete history. The purpose is to have the ability to go look back when diagnosing and issue, later index and use this to audit, replay, rebuild data, etc. For now, I just want to dump them into S3 (cheap) and later I'll dig into it when needed.

• I don't need to transform or manipulate the data, I can just dump it for now

Can I use AWS provided integrations to accomplish this or do I need to write my own HTTP service or Lambda?

I'd rather not have to write and maintain a lambda or service if I can avoid it.

What I've seen so far is that I can set an HTTP endpoint as an SNS subscriber but it has a different message format than the S3 http api thus requiring a lambda to transform it. Also the initial subscriber confirmation message. Is this not possible and I should get to work on my own service/lambda?

Answer 1

As far as I know this isn't possible.

It would be pretty trivial to write a lambda which can persist your SNS or SQS messages.

You can also use something like sqs-s3-logger (note this code is based on the fact you couldn't trigger a Lambda from SQS, this is possible now so you don't need a cron job)