Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Use case of dummy operator

Tags:

I was learning apache airflow and found that there is an operator called DummyOperator. I googled about its use case, but couldn't find anything that I can understand. Can anyone here please discuss its use case?

like image 606
Nabin Avatar asked Aug 13 '19 15:08

Nabin


People also ask

What is the use of dummy operator?

"[T]he verb do, used as an auxiliary, is often called the dummy operator because it has no meaning of its own but exists simply to fill the 'slot' of operator when an operator is needed to form (for example) negative or interrogative sentences.

What is use of dummy operator in airflow?

dummy_operator is used in BranchPythonOperator where we decide next task based on some condition. So let's suppose we have some condition in task B which decides whether to follow [task C->task D] or task E(Dummy) to reach task F.

What is a dummy operator?

The auxiliary verb do used with a main verb when forming interrogative or negative sentences, or for adding emphasis. Also called the DUMMY OPERATOR. I do not like cheese. Don't cross the road.

What is Task Group in Airflow?

Unlike SubDAGs, Task Groups are just a UI grouping concept. Starting in Airflow 2.0, you can use Task Groups to organize tasks within your DAG's graph view in the Airflow UI. This avoids the added complexity and performance issues of SubDAGs, all while using less code!


2 Answers

Operator that does literally nothing. It can be used to group tasks in a DAG.

https://airflow.apache.org/_api/airflow/operators/dummy_operator/index.html

as far as I know, at least to two case:

  • test purpose. in dags, the dummy operation just between upstream and downstream, later, you can replace the true operator.
  • Workflow purpose: BranchPythonOperator work with DummyOperator. If you want to skip some tasks, keep in mind that you can’t have an empty path, if so make a dummy task. https://airflow.apache.org/concepts.html#workflows
like image 179
Yong Wang Avatar answered Sep 21 '22 09:09

Yong Wang


dummy_operator is used in BranchPythonOperator where we decide next task based on some condition.

For example:

                  -> task C->task D

 task A -> task B                      -> task F
                   ->  task E(Dummy)

So let's suppose we have some condition in task B which decides whether to follow [task C->task D] or task E(Dummy) to reach task F.

Since we cannot leave else condition empty we have to put dummy operator which does nothing just skip or bypass.

like image 41
Suraj Mishra Avatar answered Sep 22 '22 09:09

Suraj Mishra