How to log inside Python function in PythonOperator in Airflow

Question

I am using a PythonOperator in my Airflow DAG and I need to print something inside the operator's Python function. I tried to print but apparently it didn't work out. Not so sure that's going to work. Next I tried to pass self.log in PythonOperator but I am not sure how to pass that reference.

task = PythonOperator(
    task_id='task1',
    python_callable=my_func,
    params={ ... },
    provide_context=True,
    dag=dag
)

...

def my_func(**context):
    ...
    print(some_message) # this didn't work.

Answer 1

The way you should be logging in Airflow (and in Python in general) is through the logging module, that is,

import logging

on top of the DAG definition and

logging.info(some_message)

in place of the print statement in your my_func function. Other functions you can use besides info() (with different logging level/criticality) can be found in the Python docs: https://docs.python.org/3/howto/logging.html#logging-basic-tutorial