Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get xcom as a PostgresOperator parameter?

Tags:

airflow

I created a xcom and I would like to get the result as a PostgresOperator parameter. I tried this

my_task = PostgresOperator(
    task_id=‘my_task',
    postgres_conn_id=config.get(env, 'redshift_conn'),
    sql="my_task.sql",
    params={
            ‘my_parameter': {{ int(ti.xcom_pull(task_ids=‘previous_task')) }}
    },
    dag=dag
) 
like image 508
Corentin Duhamel Avatar asked Jun 15 '26 06:06

Corentin Duhamel


2 Answers

You need to use templating when accessing xcom within an operator.

my_task = PostgresOperator(
    task_id='my_task',
    postgres_conn_id=config.get(env, 'redshift_conn'),
    sql="my_task.sql",
    params={
            'my_parameter': "{{ti.xcom_pull(task_ids='previous_task')}}"
    },
    dag=dag
) 
like image 145
Philipp Johannis Avatar answered Jun 18 '26 08:06

Philipp Johannis


I was stuck at this problem for a long time. What finally settled for me was this article which shows you can use XCom directly into your SQL file, avoiding this "params" hassle:

... from table where id = "{{ ti.xcom_pull(task_ids='task', key='id') }}"
    task_op = PostgresOperator(
        task_id="dbop",
        postgres_conn_id='db_conn',
        sql='query.sql'
        # no need to use parameters here
    )

Either ti or task_instance is available by Jinja, along many other useful ones.

like image 44
Fernando Piancastelli Avatar answered Jun 18 '26 08:06

Fernando Piancastelli



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!