DynamoDB Streams to S3

Question

I am using Data Pipeline (DP) for daily backups of DynamoDB, however, I would like to do incremental backups of the data that is missed by DP runs (updates between DP runs). To accomplish that, I would like to use DynamoDB Streams + Lambda + S3 to bring real-time DynamoDB updates to S3. I understand how DynamoDB streams work, however, I am struggling with creating a Lambda function that writes to S3 and say rolls a file every hour.

Has anyone tried it?

Answer 1

Its an hour job dude,What you need to do is

1. Enable Dynamo DB update Stream and attach aws provided lambda function https://github.com/awslabs/lambda-streams-to-firehose
2. Enable Firehose stream and use above function to stream outs records in firehose.
3. Configure Firehose to dump the records to S3.

done.