AirFlow - disable dag after X consecutive fails

Question

I read the API reference and couldnt find anything on it, is that possible?

Answer 1

Currently, there is no such feature that does it out-of-the-box but you can write some custom code in your DAG to get around this. For example, use PythonOperator (you can use MySQL operator if your metadata db is mysql) to get status of the last X runs for the dag. use BranchPythonOperator to see if the number is more than X, if it is then use a BashOperator to run airflow pause dag cli.

You can also just make this a 2-step task by adding logic of PythonOperator in BranchPythonOperator. This is just an idea, you can use a different logic.