I created a xcom and I would like to get the result as a PostgresOperator parameter. I tried this
my_task = PostgresOperator(
task_id=‘my_task',
postgres_conn_id=config.get(env, 'redshift_conn'),
sql="my_task.sql",
params={
‘my_parameter': {{ int(ti.xcom_pull(task_ids=‘previous_task')) }}
},
dag=dag
)
You need to use templating when accessing xcom within an operator.
my_task = PostgresOperator(
task_id='my_task',
postgres_conn_id=config.get(env, 'redshift_conn'),
sql="my_task.sql",
params={
'my_parameter': "{{ti.xcom_pull(task_ids='previous_task')}}"
},
dag=dag
)
I was stuck at this problem for a long time. What finally settled for me was this article which shows you can use XCom directly into your SQL file, avoiding this "params" hassle:
... from table where id = "{{ ti.xcom_pull(task_ids='task', key='id') }}"
task_op = PostgresOperator(
task_id="dbop",
postgres_conn_id='db_conn',
sql='query.sql'
# no need to use parameters here
)
Either ti or task_instance is available by Jinja, along many other useful ones.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With