Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What start_date should I use for a manually triggered DAG?

Many of the airflow example dags that have schedule_interval=None set a dynamic start date like airflow.utils.dates.days_ago(2) or datetime.utcnow(). However, the docs recommend against a dynamic start date:

We recommend against using dynamic values as start_date, especially datetime.now() as it can be quite confusing. The task is triggered once the period closes, and in theory an @hourly DAG would never get to an hour after now as now() moves along.

Is start date irrelevant for manually triggered dags? What is the best practice here?

like image 470
rcorre Avatar asked Jun 07 '18 15:06

rcorre


2 Answers

If you have a schedule_interval=None I believe the start_date is irrelevant as airflow will not attempt to perform any back filling. Just set it to anything even if it's a dynamic one it shouldn't cause any hassle.

like image 153
Simon D Avatar answered Oct 03 '22 21:10

Simon D


I ended up just setting start_date to 1970, Jan 1st (absurdly far in the past) so that Airflow never complains that the execution date is before the start date.

like image 33
Andy Carlson Avatar answered Oct 03 '22 21:10

Andy Carlson