Provide extra information to Flask's app.logger

Question

The default debug log format for Flask 0.10 is

debug_log_format =
'-------------------------------------------------------------------------\n%
%(levelname)s in %(module)s [%(pathname)s:%(lineno)d]:\n%(message)s
\n-------------------------------------------------------------------------'

How do I change it to this:

'-------------------------------------------------------------------------\n%
work_id %(levelname)s in %(module)s [%(pathname)s:%(lineno)d]:\n%(message)s
\n-------------------------------------------------------------------------'

where work_id is a randomly generated UUID for each request.

If the logger is created by myself, I can just use a logging.LoggerAdapter and provide the extra information as a dict {'work_id': some_uuid}, then I can access it in the log record using record.work_id.

But the app.logger is created by create_logger() in Flask'slogging.py, do I have to modify the Flask source to achieve what I want?

I also thought just override app.logger with my own logger, like app.logger = my_logger, it doesn't seem right.

Answer 1

Via Flask.debug_log_format

Just do this:

app.debug = True
app.debug_log_format = """-------------------------------------------------------------------------
%(worker_id)s (levelname)s in %(module)s [%(pathname)s:%(lineno)d]:\n%(message)s
-------------------------------------------------------------------------"""
app.logger.log("test", extra={"worker_id": request.your_uuid_property)

Example:

import logging
from flask import Flask, request
app = Flask(__name__)

# please replace "request.uuid" with your actual property
log = lambda msg: app.logger.info(msg, extra={'worker_id': "request.uuid" })

@app.route("/")
def hello():
    log("hello world")
    return "Hello World!"

if __name__ == "__main__":
    app.debug_log_format = """-------------------------------------------------------------------------
    %(worker_id)s in %(module)s [%(pathname)s:%(lineno)d]:
    %(message)s
    -------------------------------------------------------------------------"""
    app.debug = True
    log("hello world")
    app.run()

Via Handler and Formatter of standard logging module

Flask uses logging any way, so you can use logging.Handler and logging.Formatter to achieve outside Flask. A generic example can be found here. Advanced topic of logging configuration can be found in the doc and in the cookbook

A tailored example regarding your question is:

import logging
from flask import Flask
app = Flask(__name__)

class CustomFormatter(logging.Formatter):
    def format(self, record):
        record.worker_id = "request.uuid" # replace this with your variable 
        return super(CustomFormatter,self).format(record)

@app.route("/")
def hello():
    app.logger.info("hello world")
    return "Hello World!"

if __name__ == "__main__":
    custom_format = """-------------------------------------------------------------------------
    %(worker_id)s in %(module)s [%(pathname)s:%(lineno)d]:
    %(message)s
    -------------------------------------------------------------------------"""
    app.debug = True
    ch = logging.StreamHandler()
    ch.setFormatter(CustomFormatter(fmt=custom_format))
    app.logger.addHandler(ch)
    app.logger.debug("hello world")
    app.run()

Via overriding logging.Logger class

The same objective can be achieved by override the default logger class. Combining the flask request context stack, you would be able to get your own field in the log:

import logging
from flask import Flask
app = Flask(__name__)
from flask import _request_ctx_stack

CUSTOM_FORMAT = """-------------------------------------------------------------------------
%(worker_id)s in %(module)s [%(pathname)s:%(lineno)d]:
%(message)s
-------------------------------------------------------------------------"""

class MyLogger(logging.Logger):
    def makeRecord(self, name, level, fn, lno, msg, args, exc_info, func=None, extra=None):
        ctx = _request_ctx_stack.top
        custom_extra = dict(
            worker_id="request.uuid"
        )
        if ctx is not None:
            url = ctx.request.url # please replace this with your own field
            custom_extra["worker_id"] = url

        if extra is not None:
            extra.update(custom_extra)
        else:
            extra = custom_extra
        return super(MyLogger,self).makeRecord(name, level, fn, lno, msg, args, exc_info, func=func, extra=extra)

logging.setLoggerClass(MyLogger)

@app.route("/")
def hello():
    app.logger.info("hello world")
    return "Hello World!"

if __name__ == "__main__":
    app.debug_log_format = CUSTOM_FORMAT
    app.debug = True
    app.logger.debug("hello world")
    app.run()

Answer 2

Here is another example using a custom Formatter. Thanks to @chfw and to this

I like the use of flask.has_request_context() here, so that logging doesn't get in the way of unit tests

import logging
from logging import StreamHandler
import flask
from flask import Flask, g, request

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
app = flask.Flask(__name__)


class CustomFormatter(logging.Formatter):
    def format(self, record):
        record.uuid = None
        if flask.has_request_context():
            record.uuid = g.uuid if hasattr(g, 'uuid') else None
            record.path = request.path
            record.endpoint = request.endpoint
            record.remote_addr = request.remote_addr
        return super(CustomFormatter, self).format(record)

custom_format = '''%(levelname)s %(name)s %(uuid)s %(path)s %(endpoint)s %(remote_addr)s  %(message)s'''
handler = StreamHandler()
handler.setFormatter(CustomFormatter(fmt=custom_format))
logger.addHandler(handler)


with app.test_request_context():
    g.uuid = 'foo'
    logger.fatal('help')