Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python Script as a Cron on AWS S3 buckets

I have a python script which copy files from one S3 bucket to another S3 bucket. This script needs to be run every Sunday at some specific time. I was reading some of articles and answers, So I tried to use AWS lambda + Cloudwatch events. This files run for minimum 30 minutes. would it be still good with Lambda as Lambda can run max 15 minutes only. Or is there any other way? I can create an EC2 box and run it as a Cron but that would be expensive. Or any other standard way?

like image 763
Jaishree Mishra Avatar asked Oct 12 '25 07:10

Jaishree Mishra


1 Answers

The more appropriate way would be to use aws glue python shell job as it is under the serverless umbrella and you'll be charged as you go.

So this way you will only be charged for the time your code runs. Also you don't need to manage the EC2 for this. This is like an extended lambda.

like image 82
Shubham Jain Avatar answered Oct 14 '25 20:10

Shubham Jain