Menu
I am new at airflow and when i click run 'ignore all dependence' on Task Instance Context Menu like this:
Task Instance Context Menu
It leads to 'Only works with the CeleryExecutor'
I try to Refresh the Web UI but it doesn't work.
(I use LocalExecutor and don't want to use CeleryExecutor)
Why it happened and how can i run a single task ignore all dependence on the Web UI when i use LocalExecutor
901
CeleryExecutor is one of the ways you can scale out the number of workers. For this to work, you need to setup a Celery backend (RabbitMQ, Redis, …) and change your airflow. cfg to point the executor parameter to CeleryExecutor and provide the related Celery settings.
There are two types of executor - those that run tasks locally (inside the scheduler process), and those that run their tasks remotely (usually via a pool of workers).
Airflow comes with various executors, but the most widely used among those is Airflow Celery Executor used for scaling out by distributing the workload on multiple Celery workers that can run on different machines. CeleryExecutor works with some workers it has to distribute the tasks with the help of messages.
Once the scheduled time arrives, the Airflow scheduler sends the command to the Executor. After received signals from the scheduler, the Executor starts allocating resources and put tasks into the queue. When work is available, it will pick up the tasks from the queue to execute it.
I had a similar problem. Issue was following:
With LocalExecutor you cannot run single task, you could only run the whole DAG at once. Source code
DAG was already in 'success' state.
Possible solution is to change DAG status to running.
161
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
