How to schedule a job to execute Python script in cloud to load data into bigquery?

Question

I am trying to setup a schedule job/process in cloud to load csv data into Bigquery from google buckets using a python script. I have manage to get hold off the python code to do this but not sure where do I need to save this code so that this task could be completed as an automated process rather than running the gsutil commands manualy.

Answer 1

Reliable Task Scheduling on Google Compute Engine  |  Solutions  |  Google Cloud Platform, the 1st link in Google on "google cloud schedule a cron job", gives a high-level overview. Scheduling Tasks With Cron for Python  |  App Engine standard environment for Python  |  Google Cloud Platform, the 2nd link, has step-by-step instructions. They boil down to:

• Create cron.yaml in the specificed format alongside your app.yaml
• optionally test it at a development server
• upload it to the Google Cloud with appcfg.py update or update_cron