Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using python logging package, how to insert additional formatting into multiple logger handlers that have their own formatters?

I have a single logger with multiple handlers, which have their own formatters. Now I want to add an indenting feature, with the indent level controlled at runtime. I want messages from all the handlers to get this indent. I've tried to create it as a filter, but found that I seem to be unable to alter the message content. Then I've tried it as a formatter, but I can only have one per handler. How can I add such indentation without explicitly changing the formatters of every handler?
I should mention that one of the formatters I have is a class that adds color to the output. It is not a simple format string.


In addition, I am using a config file. Ideally, I'd like to be able to drive this mostly from there. However, I need to modify the state of the indent formatter (e.g. set indent level), but I don't know how to get to that particular formatter as there's no logger.getFormatter("by_name") method.
To clarify, I need to access the specific formatter instance, essentially to adjust the format on the fly. The instance has been created by logging.config from the file. I don't find any accessor methods that would allow me to get the particular formatter given its name.

like image 444
Evgen Avatar asked Jan 20 '11 02:01

Evgen


People also ask

How do I create a multiple logging level in Python?

You can set a different logging level for each logging handler but it seems you will have to set the logger's level to the "lowest". In the example below I set the logger to DEBUG, the stream handler to INFO and the TimedRotatingFileHandler to DEBUG. So the file has DEBUG entries and the stream outputs only INFO.

What is Handler in logging Python?

Python Logging Handler The log handler is the component that effectively writes/displays a log: Display it in the console (via StreamHandler), in a file (via FileHandler), or even by sending you an email via SMTPHandler, etc. Each log handler has 2 important fields: A formatter which adds context information to a log.


2 Answers

#!/usr/bin/env python

import logging
from random import randint

log = logging.getLogger("mylog")
log.setLevel(logging.DEBUG)

class MyFormatter(logging.Formatter):
    def __init__(self, fmt):
        logging.Formatter.__init__(self, fmt)

    def format(self, record):
        indent = " " * randint(0, 10) # To show that it works
        msg = logging.Formatter.format(self, record)
        return "\n".join([indent + x for x in msg.split("\n")])

# Log to file
filehandler = logging.FileHandler("indent.txt", "w")
filehandler.setLevel(logging.DEBUG)
filehandler.setFormatter(MyFormatter("%(levelname)-10s %(message)s"))
log.addHandler(filehandler)

# Log to stdout too
streamhandler = logging.StreamHandler()
streamhandler.setLevel(logging.INFO)
streamhandler.setFormatter(MyFormatter("%(message)s"))
log.addHandler(streamhandler)

# Test it
log.debug("Can you show me the dog-kennels, please")
log.info("They could grip it by the husk")
log.warning("That's no ordinary rabbit!")
log.error("Nobody expects the spanish inquisition")
try:
    crunchy_frog()
except:
    log.exception("It's a real frog")

result:

    They could grip it by the husk
    That's no ordinary rabbit!
          Nobody expects the spanish inquisition
         It's a real frog
         Traceback (most recent call last):
           File "./logtest2.py", line 36, in 
             crunchy_frog()
         NameError: name 'crunchy_frog' is not defined

I'm not sure I understand your second question.

like image 52
Lauritz V. Thaulow Avatar answered Oct 04 '22 21:10

Lauritz V. Thaulow


Here's another, hacky, but simple one. My messages for all handlers always start with a message level string. Just modify those darn strings on every indent change:

# (make a LEVELS dict out of all the logging levels first)    
def indent(self, step = 1):
        "Change the current indent level by the step (use negative to decrease)"
        self._indent_level += step
        if self._indent_level < 0:
            self._indent_level = 0
        self._indent_str = self._indent_str_base * self._indent_level
        for lvl in LEVELS:
            level_name = self._indent_str + LEVELS[lvl]
            logging.addLevelName(lvl, level_name)

(see my other answer for the stuff that surrounds indent function)
Now the indenter can be an independent class without messing with the details of logging process. As long as the message includes the level string, the indent will be there, even if some stuff is coming before it. Not ideal in general, but may work for me.
Anybody has more ideas that work for any msg format?

like image 24
Evgen Avatar answered Oct 04 '22 23:10

Evgen