Macros in the Airflow Python operator

Question

Can I use macros with the PythonOperator? I tried following, but I was unable to get the macros rendered:

dag = DAG(     'temp',     default_args=default_args,     description='temp dag',     schedule_interval=timedelta(days=1))  def temp_def(a, b, **kwargs):     print '{{ds}}'     print '{{execution_date}}'     print 'a=%s, b=%s, kwargs=%s' % (str(a), str(b), str(kwargs))  ds = '{{ ds }}' mm = '{{ execution_date }}'  t1 = PythonOperator(     task_id='temp_task',     python_callable=temp_def,     op_args=[mm , ds],     provide_context=False,     dag=dag) 

Answer 1

Macros only get processed for templated fields. To get Jinja to process this field, extend the PythonOperator with your own.

class MyPythonOperator(PythonOperator):     template_fields = ('templates_dict','op_args') 

I added 'templates_dict' to the template_fields because the PythonOperator itself has this field templated: PythonOperator

Now you should be able to use a macro within that field:

ds = '{{ ds }}' mm = '{{ execution_date }}'  t1 = MyPythonOperator(     task_id='temp_task',     python_callable=temp_def,     op_args=[mm , ds],     provide_context=False,     dag=dag)