Logging in python with YAML and filter

Question

Want to set up a logger with a filter using YAML.

YAML configuration file config.yaml is as follows:

version: 1

formatters:
  simple:
    format: "%(asctime)s %(name)s: %(message)s"
  extended:
    format: "%(asctime)s %(name)s %(levelname)s: %(message)s"

filters:
  noConsoleFilter:
    class: noConsoleFilter

handlers:
  console:
    class: logging.StreamHandler
    level: INFO
    formatter: simple
    filters: [noConsoleFilter]

  file_handler:
    class: logging.FileHandler
    level: INFO
    filename: test.log
    formatter: extended

root:
  handlers: [console, file_handler]
  propagate: true

...and the main program in main.py as follows:

import logging.config
import yaml

class noConsoleFilter(logging.Filter):
    def filter(self, record):
        print("filtering!")
        return not (record.levelname == 'INFO') & ('no-console' in record.msg)

with open('config.yaml', 'r') as f:
    log_cfg = yaml.safe_load(f.read())
    logging.config.dictConfig(log_cfg)  

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

logger.info("no-console. Should not be in console, but be in test.log!")
logger.info('This is an info message')
logger.error('This is an error message')

Expected output in console without the "no-console" message:

   2020-04-27 18:05:26,936 __main__: This is an info message 
   2020-04-27 18:05:26,936 __main__: This is an error message

But looks like class: noConsoleFilter is not even being considered, as the print statement is also not working.

Where am I going wrong? How can I fix it?

Answer 1

The syntax is a bit odd, but it's described in the logging docs, under User-defined objects, that you have to use the key (), not class. Like so:

filters:
  noConsoleFilter:
    (): noConsoleFilter

Next, you need to specify a qualified name for the class. If you're running the script directly, not as a module, you can refer to it under __main__:

filters:
  noConsoleFilter:
    (): __main__.noConsoleFilter

I would also recommend using the PEP 8 CapWords convention for class names. Here's a slightly tidied up, fully self-contained example:

# logging.yml
version: 1

formatters:
  simple_formatter:
    format: "%(asctime)s %(name)s: %(message)s"
  extended_formatter:
    format: "%(asctime)s %(name)s %(levelname)s: %(message)s"

filters:
  no_console_filter:
    (): __main__.NoConsoleFilter

handlers:
  console_handler:
    class: logging.StreamHandler
    level: INFO
    formatter: simple_formatter
    filters: [no_console_filter]

  file_handler:
    class: logging.FileHandler
    level: INFO
    filename: test.log
    formatter: extended_formatter

root:
  handlers: [console_handler, file_handler]
  propagate: true

# script.py
import logging.config
import yaml

class NoConsoleFilter(logging.Filter):
    def filter(self, record):
        print('filtering!')
        return not (record.levelname == 'INFO') & ('no-console' in record.msg)

with open('logging.yml', 'r') as f:
    log_cfg = yaml.safe_load(f.read())
    logging.config.dictConfig(log_cfg)

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

logger.info('no-console. Should not be in console, but be in test.log!')
logger.info('This is an info message')
logger.error('This is an error message')