Passing arguements using Taskflow API in Airflow 2.0

Question

I am passing arguments to a Taskflow based Dag using the REST API. Looking at similar issues raised on this forum, it seems below are the usual ways of accessing the passed arguments.

#From inside a template field or file:
{{ dag_run.conf['key'] }}
#Or when context is available, e.g. within a python callable of the PythonOperator:
context['dag_run'].conf['key'] 

I attempted to get the context dictionary

@dag(default_args=default_args, schedule_interval=None, start_date=days_ago(2), params=None)
def classic_xgb(**context):
    """
    ### TaskFlow API Tutorial Documentation
    [here](https://airflow.apache.org/docs/stable/tutorial_taskflow_api.html)
    """
    @task()
    def extract():
        print("context is ", context)

The output is <airflow.models.dagparam.DagParam object at 0x7f735c634510> Now how do I get to the conf dictionary that was passed as the input arguement to the Dag? I need to use the arguements in my python code and so templating option does not seem viable for me.

Any help would be most appreciated.

Thanks

Best Regards,

Adeel

Answer 1

There is a new function get_current_context() to fetch the context in Airflow 2.0. Once you have the context dict, the 'params' key contains the arguments sent to the Dag via REST API. The following code solved the issue.

from airflow.operators.python import task, get_current_context

default_args = {
    'owner': 'airflow',
}
@dag(default_args=default_args, schedule_interval=None, start_date=days_ago(2))
def classic_xgb(**kwargs):

    """
    @task()
    def extract():
        context = get_current_context()