Python logging handler to append to list

Question

Somewhat similar to this (unanswered) question here:

https://stackoverflow.com/questions/33136398/python-logging-handler-that-appends-messages-to-a-list

I am looking to use a handler to simply append anything (passing a filter) to a python list. I would envisage some kind of code like this:

import logging
import sys
mylog = logging.getLogger()
mylog.setLevel(logging.DEBUG)

log_list = []
lh = logging.SomeHandler(log_list)
lh.setLevel(logging.DEBUG)

mylog.addHandler(lh)
mylog.warning('argh')
print log_list[0]

The question is therefore - how can I implement this? What SomeHandler can I use?

Answer 1

Here is a naive, non thread-safe implementation:

import logging

class ListHandler(logging.Handler): # Inherit from logging.Handler
        def __init__(self, log_list):
                # run the regular Handler __init__
                logging.Handler.__init__(self)
                # Our custom argument
                self.log_list = log_list
        def emit(self, record):
                # record.message is the log message
                self.log_list.append(record.msg) 

Answer 2

@imriqwe's answer is correct for a non thread-safe implementation, but if you need to be thread-safe, one solution is to use a queue.Queue() instead of a list. Here is some code I am using in an in-process project to generate a tkinter log window.

import logging
import queue

class QueuingHandler(logging.Handler):
    """A thread safe logging.Handler that writes messages into a queue object.

       Designed to work with LoggingWidget so log messages from multiple
       threads can be shown together in a single ttk.Frame.

       The standard logging.QueueHandler/logging.QueueListener can not be used
       for this because the QueueListener runs in a private thread, not the
       main thread.

       Warning:  If multiple threads are writing into this Handler, all threads
       must be joined before calling logging.shutdown() or any other log
       destinations will be corrupted.
    """

    def __init__(self, *args, message_queue, **kwargs):
        """Initialize by copying the queue and sending everything else to superclass."""
        logging.Handler.__init__(self, *args, **kwargs)
        self.message_queue = message_queue

    def emit(self, record):
        """Add the formatted log message (sans newlines) to the queue."""
        self.message_queue.put(self.format(record).rstrip('\n'))

To use, create a queue, create the handler using the queue, then add it to the logger (this example also creates a log file in the current directory):

LOG_FORMAT = '%(asctime)s: %(name)8s: %(levelname)8s: %(message)s'
#  Setup root logger to write to a log file.
logging.basicConfig(filename='gui-test.log',
                    filemode='w',
                    format=LOG_FORMAT,
                    level=logging.DEBUG
                   )

#  Get a child logger
logger = logging.getLogger(name='gui')

#  Build our QueuingHandler
message_queue = queue.Queue()
handler = QueuingHandler(message_queue=message_queue, level=logging.DEBUG)

#  Change the date/time format for the GUI to drop the date
formatter = logging.Formatter(LOG_FORMAT)
formatter.default_time_format = '%H:%M:%S'
handler.setFormatter(formatter)

#  Add our QueuingHandler into the logging heirarchy at the lower level
logger.addHandler(handler)

Now all you have to do is read your messages from the queue.