Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Triggering an Airflow DAG from terminal not working

Tags:

I'm trying to use airflow to define a specific workflow that I want to manually trigger from the command line.

I create the DAG and add a bunch of tasks.

dag = airflow.DAG(     "DAG_NAME",     start_date=datetime(2015, 1, 1),     schedule_interval=None,     default_args=args) 

I then run in the terminal

airflow trigger_dag DAG_NAME 

and nothing happens. The scheduler is running in another thread. Any direction is much appreciated. Thank You

like image 485
amustafa Avatar asked May 05 '16 01:05

amustafa


People also ask

How do you start a DAG in Airflow?

When you reload the Airflow UI in your browser, you should see your hello_world DAG listed in Airflow UI. In order to start a DAG Run, first turn the workflow on (arrow 1), then click the Trigger Dag button (arrow 2) and finally, click on the Graph View (arrow 3) to see the progress of the run.


2 Answers

I just encountered the same issue.

Assuming you are able to see your dag in airflow list_dags or via the web server then:

Not only did I have to turn on the dag in the web UI, but I also had to ensure that airflow scheduler was running as a separate process.

Once I had the scheduler running I was able to successfully execute my dag using airflow trigger_dag <dag_id>

My dag configuration is not significantly different from yours. I also have schedule_interval=None

like image 63
neolytics Avatar answered Sep 21 '22 04:09

neolytics


You may have disabled the workflow. To enable the workflow manually. Open up the airflow web server by

$ airflow webserver -p 8080 

Go to http://localhost:8080 . You should see the list of all available dags with a toggle button on/off. By default everything is set to off. Search for your dag and toggle your workflow. Now try triggering the workflow from terminal. It should work now.

like image 42
sidd607 Avatar answered Sep 19 '22 04:09

sidd607