Writing to Airflow Logs

Question

One way to write to the logs in Airflow is to return a string from a PythonOperator like on line 44 here.

Are there other ways that allow me to write to the airflow log files? I've found that print statements are not saved to the logs.

Answer 1

You can import the logging module into your code and write to logs that way

import logging

logging.info('Hello')

Here are some more options

import logging    

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

Answer 2

You might use airflow logger

import logging


logger = logging.getLogger("airflow.task")
logger.error("Your custom error")

Answer 3

There's a logger mixin class you can use:

https://github.com/apache/airflow/blob/main/airflow/utils/log/logging_mixin.py

from airflow.utils.log.logging_mixin import LoggingMixin

LoggingMixin().log.info("Hello")

Then in airflow logs you'll see:

[2021-07-07 15:55:42,370] {{logging_mixin.py:112}} INFO - Hello