Menu
I have a DAG in airflow and for now it is running each hour (@hourly). Is it possible to have it running each 5 minutes ?
693
Note that DAG Runs can also be created manually through the CLI while running an airflow trigger_dag command, where you can define a specific run_id . The DAG Runs created externally to the scheduler get associated to the trigger's timestamp, and will be displayed in the UI alongside scheduled DAG runs .
Setting a cron-based schedule You can pass any cron expression as a string to the schedule parameter in your DAG. For example, if you want to schedule your DAG at 4:05 AM every day, you would use schedule='5 4 * * *' . If you need help creating the correct cron expression, see crontab guru.
Each DAG run in Airflow has an assigned “data interval” that represents the time range it operates in. For a DAG scheduled with @daily , for example, each of its data interval would start each day at midnight (00:00) and end at midnight (24:00).
Yes, here's an example of a DAG that I have running every 5 min:
dag = DAG(dag_id='eth_rates',
default_args=args,
schedule_interval='*/5 * * * *',
dagrun_timeout=timedelta(seconds=5))
schedule_interval accepts a CRON expression: https://en.wikipedia.org/wiki/Cron#CRON_expression
131
The documentation states:
Each DAG may or may not have a schedule, which informs how DAG Runs are created. schedule_interval is defined as a DAG arguments, and receives preferably a cron expression as a str, or a datetime.timedelta object.
When following the provided link for CRON expressions it appears you can specify it as */5 * * * * to run it every 5 minutes.
I'm not familiar on the matter, but this is what the documentation states.
37
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
