Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Accessing the 'ds' variable in airflow

Tags:

airflow

I am able to access the macros in python code like below:

partition_dt = macros.ds_add(ds, 1)

But i am not able to figure out how to get hold of the ds variable itself which seemingly can only be accessed in templates. Any pointers?

like image 687
Pratik Khadloya Avatar asked Mar 31 '17 20:03

Pratik Khadloya


1 Answers

I assume you want to call one of the default variables built-in AirFlow ds - the execution date as YYYY-MM-DD

To call just ds, you can do:

EXEC_DATE = '{{ ds }}'

To call what you wanted - macros.ds_add:

EXEC_DATE = '{{ macros.ds_add(ds, 1) }}'

And load it this way:

T1 = BashOperator(\
        task_id='test_ds',
        bash_command='echo ' + EXEC_DATE
        dag=DAG)

In case you want to format it (like I had to), you can do:

EXEC_DATE = '{{ macros.ds_format(macros.ds_add(ds, 1), "%Y-%m-%d", "%Y%m%d") }}'
like image 167
Oleg Yamin Avatar answered Nov 11 '22 15:11

Oleg Yamin