Menu
The goal is pretty simple: I need to create a DAG for a manual task that should not run periodically, but only when admin presses the "Run" button. Ideally without a need to switch "unpause" and "pause" the DAG (you know someone will surely forget to pause).
So far I only came with schedule_interval="0 0 30 2 *" (30th Feb hopefully never occurs), but there must be a better way!
Is there?
885
To dynamically create tasks, you need to call the method expand. This is a new method available for any operator that multiplies the task that calls for it. It expects a dict, a list, or one of those types stored in XCom as the result of a task. That method works for both the task decorators and the classic operators.
To schedule a dag, Airflow just looks for the last execution date and sum the schedule interval . If this time has expired it will run the dag. You cannot simple update the start date. A simple way to do this is edit your start date and schedule interval , rename your dag (e.g. xxxx_v2.py) and redeploy it.
A Task is the basic unit of execution in Airflow. Tasks are arranged into DAGs, and then have upstream and downstream dependencies set between them into order to express the order they should run in.
Based on the documentation, you can set the scheduler preset to None (Don’t schedule, use for exclusively “externally triggered” DAGs). Also, you can set it to @once if schedule once and only once.
82
Set schedule_interval=None.
For example:
from airflow import models
with models.DAG(
'Your DAG',
schedule_interval=None,
start_date=datetime(2021, 1, 1)
) as dag:
...
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
