Using custom formatter classes with Python's logging.config module

Question

I have the following logging class, which works fine when assigned as a formatter in code. It extends an existing formatter by prepending a string to the start of the message to be logged, to help show the importance of the message. (I don't just use %(levelname)s in the format string because I don't want to show DEBUG or INFO prefixes.)

class PrependErrorLevelFormatter(logging.Formatter):
    def __init__(self, default):
        self._default_formatter = default
    def format(self, record):
        if record.levelno == logging.WARNING:
            record.msg = "[WARNING] " + record.msg
        elif record.levelno == logging.ERROR:
            record.msg = "[ERROR] " + record.msg
        elif record.levelno == logging.CRITICAL:
            record.msg = "[CRITICAL] " + record.msg
        return self._default_formatter.format(record)

Now I want to be able to assign it via a configuration file loaded in by logging.config.fileConfig(). I have tried syntax like this:

[formatter_PrependErrorLevelFormatter]
format=%(asctime)s  %(message)s
datefmt=%X
class=PrependErrorLevelFormatter

Unfortunately I get errors resolving this class:

  File "C:\Python27\lib\logging\config.py", line 70, in fileConfig
    formatters = _create_formatters(cp)
  File "C:\Python27\lib\logging\config.py", line 127, in _create_formatters
    c = _resolve(class_name)
  File "C:\Python27\lib\logging\config.py", line 88, in _resolve
    found = __import__(used)
ImportError: No module named PrependErrorLevelFormatter

I have tried prefixing the class name with the name of the module it is in, but get the same error. And even if it could resolve the class, it will probably not work due to the extra default formatter argument I need to provide.

How can I achieve the result I want using the logging.config system?

Answer 1

As you are using Python 2.7, you could use dictionary-based configuration using dictConfig(): this is more flexible than fileConfig(), as it allows use of arbitrary callables as factories to return e.g. handlers, formatters, or filters.

If you have to use fileConfig(), you'll have to construct a callable which takes format and datefmt string values and returns an instance of your class. The class value just needs to resolve to a callable, not an actual class. Here's a setup that works: In this gist, I have a file custfmt.py which contains the formatter definition, and a script fcfgtest.py which uses it via fileConfig(). Just get the files into a scratch directory and run fcfgtest.py - you should see output like this:

20:17:59 debug message
20:17:59 info message
20:17:59 [WARNING] warning message
20:17:59 [ERROR] error message
20:17:59 [CRITICAL] critical message

which appears to be what you need.

Note that you can use an alternative design for your formatter, which should do the same job:

class AltCustomFormatter(logging.Formatter):
    def format(self, record):
        if record.levelno in (logging.WARNING,
                              logging.ERROR,
                              logging.CRITICAL):
            record.msg = '[%s] %s' % (record.levelname, record.msg)
        return super(AltCustomFormatter , self).format(record)

To use this, you don't need a separate factory function, so you can just use

class=custfmt.AltCustomFormatter

instead of

class=custfmt.factory

and it should work - it did for me when I tested just now using Python 2.7.1 :-)