How to trigger airflow dag manually?

Question

I have beening working on Airflow for a while for no problem withe the scheduler but now I have encountered a problem.

Bascially I have a script and dag ready for a task, but the task doesn't run periodically. Instead it needs to be activated at random time. (External parties will tell us it's time and we will run it. This may happen for many times in the following months.)

Is there anyway to trigger the dag manually? Any other directions/suggestions are welcomed as well. Thanks.

Answer 1

You have a number of options here:

• UI: Click the "Trigger DAG" button either on the main DAG or a specific DAG.
• CLI: Run airflow trigger_dag <dag_id>, see docs in https://airflow.apache.org/docs/stable/cli.html#trigger_dag. Note that later versions of airflow use the syntaxairflow dags trigger <dag_id>
• API: Call POST /api/experimental/dags/<dag_id>/dag_runs, see docs in https://airflow.apache.org/docs/stable/api.html#post--api-experimental-dags--DAG_ID--dag_runs.
• Operator: Use the TriggerDagRunOperator, see docs in https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/operators/trigger_dagrun/index.html#airflow.operators.trigger_dagrun.TriggerDagRunOperator and an example in https://github.com/apache/airflow/blob/master/airflow/example_dags/example_trigger_controller_dag.py.

You'll probably go with the UI or CLI if this is truly going to be 100% manual. The API or Operator would be options if you are looking to let the external party indirectly trigger it themselves. Remember to set schedule_interval=None on the DAG.

Answer 2

So a dag can be triggered by following ways:

• Using the REST API Reference(see documentation)

  endpoint-

  > POST   /api/experimental/dags/<DAG_ID>/dag_runs
  

  1. Using Curl:

     curl -X POST
     http://localhost:8080/api/experimental/dags/<DAG_ID>/dag_runs
     -H 'Cache-Control: no-cache'
     -H 'Content-Type: application/json'
     -d '{"conf":"{"key":"value"}"}'

  2. Using Python requests:

        import requests
        response = requests.post(url, data=json.dumps(data), headers=headers)
     

• Using the trigger DAG option present in the UI as mentioned by @Daniel

Answer 3

Airflow has API. The method you need is POST /api/experimental/dags/<DAG_ID>/dag_runs. With this method you also could pass config params for the dag run.

We use Jenkins to trigger dags manually. If you are using Jenkins you could check our jenkins pipeine library.