Duplicate log output when using Python logging module

Question

The logging.getLogger() is returns the same instance for a given name. (Documentation)

The problem is that every time you call myLogger(), it's adding another handler to the instance, which causes the duplicate logs.

Perhaps something like this?

import os
import time
import datetime
import logging

loggers = {}

def myLogger(name):
    global loggers
    
    if loggers.get(name):
        return loggers.get(name)
    else:
        logger = logging.getLogger(name)
        logger.setLevel(logging.DEBUG)
        now = datetime.datetime.now()
        handler = logging.FileHandler(
            '/root/credentials/Logs/ProvisioningPython' 
            + now.strftime("%Y-%m-%d") 
            + '.log')
        formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
        handler.setFormatter(formatter)
        logger.addHandler(handler)
        loggers[name] = logger
                       
        return logger

Since Python 3.2 you can just check if handlers are already present and if so, clear them before adding new handlers. This is pretty convenient when debugging and the code includes your logger initialization

if (logger.hasHandlers()):
    logger.handlers.clear()

logger.addHandler(handler)

I already used the logger as a Singleton and checked if not len(logger.handlers), but still got duplicates: It was the formatted output, followed by the unformatted.

Solution in my case: logger.propagate = False

Credits to this answer and the docs.

import datetime
import logging
class Logger :
    def myLogger(self):
       logger=logging.getLogger('ProvisioningPython')
       if not len(logger.handlers):
          logger.setLevel(logging.DEBUG)
          now = datetime.datetime.now()
          handler=logging.FileHandler('/root/credentials/Logs/ProvisioningPython'+ now.strftime("%Y-%m-%d") +'.log')
          formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
          handler.setFormatter(formatter)
          logger.addHandler(handler)
        return logger

made the trick for me

using python 2.7

You are calling Logger.myLogger() more than once. Store the logger instance it returns somewhere and reuse that.

Also be advised that if you log before any handler is added, a default StreamHandler(sys.stderr) will be created.

This is an addition to @rm957377's answer but with an explanation of why this is happening. When you run a lambda function in AWS, they call your function from within a wrapping instance that stays alive for multiple calls. Meaning, if you call addHandler() within the code of your function, it will continue to add duplicate handlers to the logging singleton every time the function runs. The logging singleton persists through multiple calls of you lambda function.

To solve this you can clear your handlers before you set them via:

logging.getLogger().handlers.clear()
logging.getLogger().addHandler(...)