Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I stop an airflow DAG?

Running Airflow 1.9.0 with python 2.7. How do I gracefully stop a DAG?

In this case, I have a DAG that's running a file upload with bad code that causes everything to take 4 times as long, and I'd really prefer not to have to wait a day for it to finally time out (timeout is set to 10 hours).

The DAG looks for a tar file. When it finds one, it goes through every file in the tar, looking for files to process, and processes them.

I could not find any way to stop the DAG. I tried clicking on the "Running" circle in the "DAG Runs" column (the one to the right). It let me select the process and mark it as "failed". But it didn't stop running.

I tried clicking on the "Running" circle in the "Recent Tasks" column (the one to the left). It let me select processes, but trying to set them to filed (or to success), generated an exception in Airflow

like image 565
Greg Dougherty Avatar asked Feb 28 '18 22:02

Greg Dougherty


2 Answers

Browse -> DAG Runs -> Checkbox -> With Selected "Delete"

like image 50
Aclwitt Avatar answered Sep 24 '22 19:09

Aclwitt


If you constructed your process the way it sounds like you did, you won't be able to stop it from Airflow. You'll need to find the process identifier that is executing and forcibly terminate it to get it to actually stop.

like image 35
joebeeson Avatar answered Sep 20 '22 19:09

joebeeson