logging with filters

Question

I'm using Logging (import logging) to log messages.

Within 1 single module, I am logging messages at the debug level my_logger.debug('msg');

Some of these debug messages come from function_a() and others from function_b(); I'd like to be able to enable/disable logging based on whether they come from a or from b;

I'm guessing that I have to use Logging's filtering mechanism.

Can someone show me how the code below would need to be instrumented to do what I want?

import logging logger = logging.getLogger( "module_name" )  def function_a( ... ):     logger.debug( "a message" )  def function_b( ... ):     logger.debug( "another message" )  if __name__ == "__main__":     logging.basicConfig( stream=sys.stderr, level=logging.DEBUG )      #don't want function_a()'s noise -> ....     #somehow filter-out function_a's logging     function_a()      #don't want function_b()'s noise -> ....     #somehow filter-out function_b's logging     function_b() 

If I scaled this simple example to more modules and more funcs per module, I'd be concerned about lots of loggers;

Can I keep it down to 1 logger per module? Note that the log messages are "structured", i.e. if the function(s) logging it are doing some parsing work, they all contain a prefix logger.debug("parsing: xxx") - can I somehow with a single line just shut-off all "parsing" messages (regardless of the module/function emitting the message?)

Answer 1

Just implement a subclass of logging.Filter: http://docs.python.org/library/logging.html#filter-objects. It will have one method, filter(record), that examines the log record and returns True to log it or False to discard it. Then you can install the filter on either a Logger or a Handler by calling its addFilter(filter) method.

Example:

class NoParsingFilter(logging.Filter):     def filter(self, record):         return not record.getMessage().startswith('parsing')  logger.addFilter(NoParsingFilter()) 

Or something like that, anyway.

Answer 2

Do not use global. It's an accident waiting to happen.

You can give your loggers any "."-separated names that are meaningful to you.

You can control them as a hierarchy. If you have loggers named a.b.c and a.b.d, you can check the logging level for a.b and alter both loggers.

You can have any number of loggers -- they're inexpensive.

The most common design pattern is one logger per module. See Naming Python loggers

Do this.

import logging  logger= logging.getLogger( "module_name" ) logger_a = logger.getLogger( "module_name.function_a" ) logger_b = logger.getLogger( "module_name.function_b" )  def function_a( ... ):     logger_a.debug( "a message" )  def function_b( ... ):     logger_b.debug( "another message" )  if __name__ == "__main__":     logging.basicConfig( stream=sys.stderr, level=logging.DEBUG )     logger_a.setLevel( logging.DEBUG )     logger_b.setLevel( logging.WARN )      ... etc ...