Python Logging: Inherit contextual information

Question

Consider the following logging example. There are two python files, myapp.py and mylib.py.

# myapp.py
import logging
import mylib

class customAdapter(logging.LoggerAdapter):
    def process(self, msg, kwargs):
        return '[%s] %s' % (self.extra['connid'], msg), kwargs

def main():
    logging.basicConfig(filename='myapp.log', level=logging.INFO)
    logger = logging.getLogger('test')
    adapter = customAdapter(logger,{'connid': 123})
    adapter.info('Started')
    mylib.do_something()
    adapter.info('Finished')

if __name__ == '__main__':
    main()

---

# mylib.py
import logging

def do_something():
    lgr = logging.getLogger('test')
    lgr.info('Doing something')

When you execute myapp.py, you get the following output to the log file:

INFO:test:[123] Started
INFO:test:Doing something
INFO:test:[123] Finished

I'm trying to get the log in inherit the contextual information acquired from the parent code. Desired output:

INFO:test:[123] Started
INFO:test:[123] Doing something
INFO:test:[123] Finished

Is this possible? (notice that "[123]" is included in line 2 of the log)

Here are a few other relevant pieces of information:

• I'm using python 3.5
• I'm trying to avoid passing any arguments to the "do_something" function.
• In actual use, the value for "connid" will be dynamic.
• This will be used in a django view. Replace main() for the django view.

Answer 1

I took Vinay Sajip's answer and applied it and got the following. It appears to be working the way I want it to. Hopefully this will help someone else as well. Thanks Vinay!

You can't easily do this for third-party libraries (as they won't use adapters, even if you do in your code). However, you can do this using filters, as described in the documentation here.

# myapp.py
import logging
import mylib


class ContextFilter(logging.Filter):
    def __init__(self, filter_name, extra):
        super(ContextFilter, self).__init__(filter_name)
        self.connid = extra

    def filter(self, record):
        record.connid = self.connid
        return True


def main():
    logging.basicConfig(filename='myapp.log',level=logging.INFO,
                        format='%(levelname)s:%(name)s:[%(connid)s] %(message)s')
    logger = logging.getLogger('test')
    cf = ContextFilter(filter_name='add_conn_id', extra='123')
    logger.addFilter(cf)
    logger.info('Started')
    mylib.do_something()
    logger.info('Finished')

if __name__ == '__main__':
    main()

My log output now looks like this:

INFO:test:[123] Started
INFO:test:[123] Doing something
INFO:test:[123] Finished

The formatting change

Answer 2

You can't easily do this for third-party libraries (as they won't use adapters, even if you do in your code). However, you can do this using filters, as described in the documentation here.