How do I implement logging on a per-thread basis?

Question

I want to get the logs from an individual thread into a specific file that only contains logs from that thread.

In most cases, it would be totally satisfactory to just log from all threads into a single file. This will not work for me. I am applying a concurrent model to a large and currently iterative code base that has logging implemented all over the place in different modules, and I am trying to change as little code as possible.

I can think of several solutions that just seem a bit hacky that involve detecting if the current code is executing in a thread, so I am wondering if there is a nice Pythonic way to do this.

Answer 1

If you're already using the logging module, its very easy. You should have something like this:

log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)
file_handler = logging.FileHandler('mylog.log')
file_handler.setFormatter(logging.Formatter('%(message)s'))
log.addHandler(file_handler)

log.debug('this is awesome!')

well go to your logging.Formatter and add %(thread)d and %(threadName)s and presto! You have a thread aware logger!

so again, it will look like this:

logging.Formatter('%(thread)d - %(threadName)s - %(levelname)s - %(message)s')

then.. grep it. Voila! No need to change anything else.

Here are all of them if you're interested: https://docs.python.org/2/library/logging.html#logrecord-attributes