Best way to 'listen' to s3 bucket for new file?

Question

I have connected zapier to a webhook I am listening too, which sends a JSON file into my s3 bucket.

I have some python code that I want to execute when a new file is uploaded into the bucket, in real time over the file.

What is the best way to 'listen' for the upload of this file into the s3 bucket?

Answer 1

You can setup an Amazon's CloudWatch Event to listen for when a new object is put into a S3 bucket. You can configure the event to work with a specific S3 bucket. When a new object is put into the bucket you can configure the event to trigger a Lambda function that will execute whatever custom Python code you want to run when a new object is added.

Your question is pretty broad so I can't get into specifics, but you can checkout the AWS documentation that provides examples to guide you through the process. AWS CloudWatch Documentation.

Answer 2

David here, from the Zapier Platform team.

Seems like you've already found your answer, which is great. I just wanted to plug Zapier as an option (since you had mentioned you're already using it). Our S3 integration has a "new file in bucket" trigger, which you can combine with any other step (such as a Python Code step). Additionally, you could skip the middleman and structure your zap as:

1. some trigger
2. Add file to S3
3. Run Python

And not need to worry about webhooks at all.

​Let me know if you've got any other questions!