Logging variable data with new format string

Question

I use logging facility for python 2.7.3. Documentation for this Python version say:

the logging package pre-dates newer formatting options such as str.format() and string.Template. These newer formatting options are supported...

I like 'new' format with curly braces. So i'm trying to do something like:

 log = logging.getLogger("some.logger")  log.debug("format this message {0}", 1) 

And get error:

TypeError: not all arguments converted during string formatting

What I miss here?

P.S. I don't want to use

log.debug("format this message {0}".format(1)) 

because in this case the message is always being formatted regardless of logger level.

Answer 1

EDIT: take a look at the StyleAdapter approach in @Dunes' answer unlike this answer; it allows to use alternative formatting styles without the boilerplate while calling logger's methods (debug(), info(), error(), etc).

---

From the docs — Use of alternative formatting styles:

Logging calls (logger.debug(), logger.info() etc.) only take positional parameters for the actual logging message itself, with keyword parameters used only for determining options for how to handle the actual logging call (e.g. the exc_info keyword parameter to indicate that traceback information should be logged, or the extra keyword parameter to indicate additional contextual information to be added to the log). So you cannot directly make logging calls using str.format() or string.Template syntax, because internally the logging package uses %-formatting to merge the format string and the variable arguments. There would no changing this while preserving backward compatibility, since all logging calls which are out there in existing code will be using %-format strings.

And:

There is, however, a way that you can use {}- and $- formatting to construct your individual log messages. Recall that for a message you can use an arbitrary object as a message format string, and that the logging package will call str() on that object to get the actual format string.

Copy-paste this to wherever module:

class BraceMessage(object):     def __init__(self, fmt, *args, **kwargs):         self.fmt = fmt         self.args = args         self.kwargs = kwargs      def __str__(self):         return self.fmt.format(*self.args, **self.kwargs) 

Then:

from wherever import BraceMessage as __  log.debug(__('Message with {0} {name}', 2, name='placeholders')) 

Note: actual formatting is delayed until it is necessary e.g., if DEBUG messages are not logged then the formatting is not performed at all.

Answer 2

Here is another option that does not have the keyword problems mentioned in Dunes' answer. It can only handle positional ({0}) arguments and not keyword ({foo}) arguments. It also does not require two calls to format (using the underscore). It does have the ick-factor of subclassing str:

class BraceString(str):     def __mod__(self, other):         return self.format(*other)     def __str__(self):         return self   class StyleAdapter(logging.LoggerAdapter):      def __init__(self, logger, extra=None):         super(StyleAdapter, self).__init__(logger, extra)      def process(self, msg, kwargs):         if kwargs.pop('style', "%") == "{":  # optional             msg = BraceString(msg)         return msg, kwargs 

You use it like this:

logger = StyleAdapter(logging.getLogger(__name__)) logger.info("knights:{0}", "ni", style="{") logger.info("knights:{}", "shrubbery", style="{") 

Of course, you can remove the check noted with # optional to force all messages through the adapter to use new-style formatting.

---

Note for anyone reading this answer years later: Starting with Python 3.2, you can use the style parameter with Formatter objects:

Logging (as of 3.2) provides improved support for these two additional formatting styles. The Formatter class been enhanced to take an additional, optional keyword parameter named style. This defaults to '%', but other possible values are '{' and '$', which correspond to the other two formatting styles. Backwards compatibility is maintained by default (as you would expect), but by explicitly specifying a style parameter, you get the ability to specify format strings which work with str.format() or string.Template.

The docs provide the example logging.Formatter('{asctime} {name} {levelname:8s} {message}', style='{')

Note that in this case you still can't call the logger with the new format. I.e., the following still won't work:

logger.info("knights:{say}", say="ni")  # Doesn't work! logger.info("knights:{0}", "ni")  # Doesn't work either