Using python logging package, how to insert additional formatting into multiple logger handlers that have their own formatters?

Question

I have a single logger with multiple handlers, which have their own formatters. Now I want to add an indenting feature, with the indent level controlled at runtime. I want messages from all the handlers to get this indent. I've tried to create it as a filter, but found that I seem to be unable to alter the message content. Then I've tried it as a formatter, but I can only have one per handler. How can I add such indentation without explicitly changing the formatters of every handler?
I should mention that one of the formatters I have is a class that adds color to the output. It is not a simple format string.

---

In addition, I am using a config file. Ideally, I'd like to be able to drive this mostly from there. However, I need to modify the state of the indent formatter (e.g. set indent level), but I don't know how to get to that particular formatter as there's no logger.getFormatter("by_name") method.
To clarify, I need to access the specific formatter instance, essentially to adjust the format on the fly. The instance has been created by logging.config from the file. I don't find any accessor methods that would allow me to get the particular formatter given its name.

Answer 1

#!/usr/bin/env python

import logging
from random import randint

log = logging.getLogger("mylog")
log.setLevel(logging.DEBUG)

class MyFormatter(logging.Formatter):
    def __init__(self, fmt):
        logging.Formatter.__init__(self, fmt)

    def format(self, record):
        indent = " " * randint(0, 10) # To show that it works
        msg = logging.Formatter.format(self, record)
        return "\n".join([indent + x for x in msg.split("\n")])

# Log to file
filehandler = logging.FileHandler("indent.txt", "w")
filehandler.setLevel(logging.DEBUG)
filehandler.setFormatter(MyFormatter("%(levelname)-10s %(message)s"))
log.addHandler(filehandler)

# Log to stdout too
streamhandler = logging.StreamHandler()
streamhandler.setLevel(logging.INFO)
streamhandler.setFormatter(MyFormatter("%(message)s"))
log.addHandler(streamhandler)

# Test it
log.debug("Can you show me the dog-kennels, please")
log.info("They could grip it by the husk")
log.warning("That's no ordinary rabbit!")
log.error("Nobody expects the spanish inquisition")
try:
    crunchy_frog()
except:
    log.exception("It's a real frog")

result:

    They could grip it by the husk
    That's no ordinary rabbit!
          Nobody expects the spanish inquisition
         It's a real frog
         Traceback (most recent call last):
           File "./logtest2.py", line 36, in 
             crunchy_frog()
         NameError: name 'crunchy_frog' is not defined

I'm not sure I understand your second question.

Answer 2

Here's another, hacky, but simple one. My messages for all handlers always start with a message level string. Just modify those darn strings on every indent change:

# (make a LEVELS dict out of all the logging levels first)    
def indent(self, step = 1):
        "Change the current indent level by the step (use negative to decrease)"
        self._indent_level += step
        if self._indent_level < 0:
            self._indent_level = 0
        self._indent_str = self._indent_str_base * self._indent_level
        for lvl in LEVELS:
            level_name = self._indent_str + LEVELS[lvl]
            logging.addLevelName(lvl, level_name)

(see my other answer for the stuff that surrounds indent function)
Now the indenter can be an independent class without messing with the details of logging process. As long as the message includes the level string, the indent will be there, even if some stuff is coming before it. Not ideal in general, but may work for me.
Anybody has more ideas that work for any msg format?