Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to check if task 1 fail then run task 2 in airflow?

Tags:

airflow

How to check failed task like task 1 is failed then run task 2, like if else condition.

I want to run the dependent task.

Task1 failed then how can i get that error log in a condition like if task1== failed then run task2 and else task3. I tried SSHHOOK but I am looking for a simple solution.

  with DAG(
    'airflow',
    catchup=False,
    default_args={
        'owner': 'abc',
    'start_date': datetime(2018, 4, 17),
        'schedule_interval':None,
        'depends_on_past': False,
    },   
) as dag:
    task_1 = PythonOperator(
        task_id='task_1', 
        python_callable=do(),
    )
    task_2 = PythonOperator(
        task_id='task_2',
        python_callable=do(),
    )
    task_3 = PythonOperator(
        task_id='task_3',
        python_callable=do()

    task_3.set_upstream(task_2)
    task_2.set_upstream(task_1)
like image 800
griez007 Avatar asked Apr 17 '18 06:04

griez007


1 Answers

Since there were no code examples I have to assume what your DAG might look like and what you want to do. Also, I didn't understand why you wanted to use SSHHook but again, no code examples. So here we go:

Create error task

def t2_error_task(context):

    instance = context['task_instance']
    do_stuff()

Create tasks

t1_task = PythonOperator(
    task_id='my_operator_t1',
    python_callable=do_python_stuff,
    on_failure_callback=t2_error_task,
    dag=dag
)

t3_task_success = PythonOperator(
    task_id='my_operator_t3',
    python_callable=do_python_stuff_success,
    dag=dag
)

Then set t3 upstream of t1:

t1_task >> t3_task_success 
like image 100
tobi6 Avatar answered Dec 31 '22 20:12

tobi6