How to schedule python script in google cloud without using cron jobs?

Question

I have two python scripts running once a day in my local environment. One is to fetch data and another is to format it.

Now I want to deploy those scripts to Google's cloud environment and run those once/twice a day.

Can I do that using Google Cloud Function or do I need App Engine?

Why NO cron job: Because I don't want my system/VM to run whole day (when not in use).

Can I use Cloud Composer to achieve that?

Answer 1

You can use Google Cloud Scheduler which is a fully managed enterprise-grade cron job scheduler. It allows you to schedule virtually any job, including batch, big data jobs, cloud infrastructure operations.

The first 3 jobs per month are free. Next are $0.10 per job/month.

Answer 2

Yes, you will need App Engine and the Cron service to schedule those scripts. You have some more or less straight-forward options to run those scripts on GCP:

• Deploy each one as different functions, and do something as mentioned in here: Basically deploy a simple function in App Engine that sends an HTTP request to your(s) function(s).
• Deploy each one as different handlers in App Engine, and schedule each calls.
• Deploy each one as different services and then schedule the calls, bearing in mind that, if you want to schedule a call to a different service, you must specify to which one, in your cron.yaml file, using the target keyword and the name of the service.