Add a correlation or trace ID to Python log calls

Question

Given a complex application in Python that uses standard library logging, something like:

import logging

logger = logging.getLogger("mycomponent")

# (Split over multiple Python files/folders of course)

def do_a_really_specific_thing(x):
    if isinstance(x, int):
        logger.info("I'm doing a thing! %s", x)
        return 2 * x
    else:
        logger.error("Ignoring non-integer value %s", x)
        return 0

def run_a_task(task_input):
    return do_a_really_specific_thing(task_input.x)

def process_a_transaction(req):
    y = run_a_task(req.data)
    result = y + 1
    logger.info("Processed transaction %s", req.txn_id)
    return result

Is there some standard pattern, without editing the code of the inner function do_a_really_specific_thing(), to decorate any sandard logging events raised within a context with additional information like a trace or correlation ID?

I'm thinking something along the lines of:

def process_a_transaction(req):
    with decorate_logs(txn_id=req.txn_id):
        y = run_a_task(req.data)
    return y + 1

It seems like a pretty standard idea, but the articles I've come across all seem to involve bringing in some big logging framework or other tools. For e.g. here with Datadog, here with SolarWinds AppOptics.

Answer 1

This is supported by stdlib logging out of the box and documented in the logging cookbook (part of the Python documentation). Specifically, you can use a LoggerAdapter or a logging filter to impart this type of additional information. The cookbook example relates to peer IP addresses for a network application, but it can be used in the same way for your transaction IDs.