Python Logging : how to add a custom field to LogRecord, and register a global callback to sets it's value

Question

In a Flask application, I want to add a user_id field added to every error log record that gets produced, whenever a user_id exists in the flask.session.

I came up with the solution below but it's hackish, since it doesn't allow for the format string to drive the formatting of the user_id, and since the logging API seems to provides ways to customize logging (LoggerAdapter, logging.makeRecord, etc...) I figure there has to be a cleaner way.

What would be the "python way" to do it ?

class CustomFormatter(Formatter):
    def format(self, record):
        from myapp.core import authenticationManager
        user_id = authenticationManager.current_user_id_if_authenticated()
        user_id = "unknown" if user_id is None else str(user_id)
        return super(F,self).format(record) + ", user_id" + user_id

Answer 1

You could define a custom filter which injects a value for record.user_id. Done this way, you can define a format which includes %(user_id)s just like other (standard) record attributes:

format='%(asctime)-15s %(name)-5s %(levelname)-8s user_id %(user_id)-15s: %(message)s')

and then all logging calls will automatically add the user_id value.

---

import logging

class UserIDFilter(logging.Filter):
    """
    This is a filter which injects contextual information into the log.
    """
    from myapp.core import authenticationManager

    def filter(self, record):
        user_id = authenticationManager.current_user_id_if_authenticated()
        user_id = "unknown" if user_id is None else str(user_id)
        record.user_id = user_id
        return True


logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)-15s %(name)-5s %(levelname)-8s user_id %(user_id)-15s: %(message)s')

logger = logging.getLogger(__name__)
filt = UserIDFilter()
logger.addFilter(filt)

Answer 2

The logaugment (Github page) package does this:

To use:

import logaugment

logaugment.set(logger, user_id='custom_value')

All logging calls with logger will now use the value 'custom_value' for %(user_id)s in the logging message.

You can repeatedly call logaugment.set without problems to change the value. Alternatively, you can specify a callable that will be called every time when a logging call is made. This would allow dynamically computing / retrieving the user ID (see example in package README).

To install:

pip install logaugment

(disclaimer: I developed this package)