Menu
Lets say today is 2017-10-20. I have an existing dag which is successful till today. I need to add a task with a start_date of 2017-10-01. How to make the scheduler trigger task from 2017-10-01 to 2017-10-20 automatically ?
429
Backfilling can be accomplished in Airflow using the CLI. You simply specify the DAG ID, as well as the start date and end date for the backfill period. This command runs the DAG for all intervals between the start date and end date. DAGs in your backfill interval are still rerun even if they already have DAG runs.
An Airflow DAG with a start_date , possibly an end_date , and a schedule_interval defines a series of intervals which the scheduler turns into individual DAG Runs and executes.
You can use the backfill command line tool.
airflow backfill your_dag_id -s 2017-10-01 -e 2017-10-20 -t task_name_regex
This is assuming there is already a DAG run for dates beginning from 2017-10-01. If that's not the case, make sure the DAG's start date is 2017-10-01 or earlier and that catchup is enabled.
178
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
