How to check failed task like task 1 is failed then run task 2, like if else condition.
I want to run the dependent task.
Task1 failed then how can i get that error log in a condition like if task1== failed then run task2 and else task3. I tried SSHHOOK
but I am looking for a simple solution.
with DAG(
'airflow',
catchup=False,
default_args={
'owner': 'abc',
'start_date': datetime(2018, 4, 17),
'schedule_interval':None,
'depends_on_past': False,
},
) as dag:
task_1 = PythonOperator(
task_id='task_1',
python_callable=do(),
)
task_2 = PythonOperator(
task_id='task_2',
python_callable=do(),
)
task_3 = PythonOperator(
task_id='task_3',
python_callable=do()
task_3.set_upstream(task_2)
task_2.set_upstream(task_1)
Since there were no code examples I have to assume what your DAG might look like and what you want to do. Also, I didn't understand why you wanted to use SSHHook but again, no code examples. So here we go:
Create error task
def t2_error_task(context):
instance = context['task_instance']
do_stuff()
Create tasks
t1_task = PythonOperator(
task_id='my_operator_t1',
python_callable=do_python_stuff,
on_failure_callback=t2_error_task,
dag=dag
)
t3_task_success = PythonOperator(
task_id='my_operator_t3',
python_callable=do_python_stuff_success,
dag=dag
)
Then set t3 upstream of t1:
t1_task >> t3_task_success
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With