Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Airflow DAG Multiple Runs

I have a DAG that I want to run multiple times after each successful completion. For an example I want to run it 10 times and stop. Is there a way to accomplish this? I tried looking into scheduling with CRON but it doesn't seem clean nor triggering the DAG via UI multiple times doesn't work (runs in parallel).

like image 306
Nk.Pl Avatar asked Feb 03 '26 09:02

Nk.Pl


1 Answers

I found a solution to my use case. It incorporated using depends_on_past=True (mentioned by @Hitesh Gupta) and setting your airflow.cfg file below:

# The maximum number of active DAG runs per DAG max_active_runs_per_dag = 1

This allowed us to only have one active DAG run at a time and also to not continue the next DAG run if there were failure in the previous run. This is for Airflow version 1.10.1 that I tested on.

like image 197
Nk.Pl Avatar answered Feb 05 '26 01:02

Nk.Pl



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!