Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to pass parameters from PythonOperator to HttpSensor via XComs in Airflow?

I want to send an HTTP request whose parameters depend on the result of a dependent Python callable. I am trying to use XComs for this purpose. Simplified example:

def get_index():
  return 0

get_index = PythonOperator(
  task_id='get_index',
  python_callable=get_index,
  dag=dag)

http_request = HttpSensor(
  task_id='send_http_request',
  http_conn_id=HTTP_HOST,
  endpoint=ENDPOINT,
  params={
    "index": "{{ ti.xcom_pull('get_index')  }}"
  },
  dag=dag)

get_index >> http_request

Unfortunately, after examining the options of the HTTP request I see the macro is not evaluated properly and instead of 0, {{ ti.xcom_pull('get_index') }} is sent. What might have gone wrong? Should I use the HttpOperator instead of HttpSensor?

like image 287
Momchil Peychev Avatar asked Nov 08 '22 17:11

Momchil Peychev


1 Answers

As faeder mentioned, jinja templates in params are currently not evaluated. I resolved the issue by switching to SimpleHttpOperatrs and putting the template in the data field.

like image 182
Momchil Peychev Avatar answered Nov 14 '22 23:11

Momchil Peychev