Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to re-run all failed tasks in Apache Airflow?

Tags:

airflow

I have a Apache Airflow DAG with tens of thousands of tasks and after a run, say a handful of them failed.

I fixed the bug that caused some tasks to fail and I would like to re-run ONLY FAILED TASKS. This SO post suggests using the GUI to "clear" failed task:

How to restart a failed task on Airflow

This approach works if you have a handful number of failed tasks.

I am wondering if we can bypass the GUI and do it problematically, through command line something like:

airflow_clear_failed_tasks dag_id execution_data 
like image 870
motam79 Avatar asked Dec 27 '18 13:12

motam79


1 Answers

The command to clear only failed tasks was updated. It is now (Airflow 2.0 as of March 2021):

airflow tasks clear [-s START_DATE] [-e END_DATE] --only-failed dag_id
like image 58
pgzmnk Avatar answered Oct 12 '22 15:10

pgzmnk