Python Logging: Provide log file path from main module

Question

Is there any way I can provide the filename for logger from my main module?

I am using following way, however it's not working.all the logs go to xyz.log file rather than main.log

Updated as per suggestion from nosklo

logger.py

formatter = logging.Formatter(fmt='[%(asctime)s] - {%(filename)s:%(lineno)d} %(levelname)s - %(message)s')


def _get_file_handler(file_name="xyz.log"):
    file_handler = logging.FileHandler(file_name)
    file_handler.setLevel(logging.DEBUG)
    file_handler.setFormatter(formatter)
    return file_handler


def get_logger(name):
    logger = logging.getLogger(name)
    logger.setLevel(logging.DEBUG)
    logger.addHandler(_get_file_handler())
    return logger

parser.py

log = logger.get_logger(__name__)


def parse():
    log.info("is there anyway this could go to main.log and xyz.log")

main.py

log = logger.get_logger(__name__)
if __name__ == '__main__':

    for handler in log.handlers:
        if isinstance(handler, logging.FileHandler):
            log.removeHandler(handler)
            log.addHandler(logger._get_file_handler())

    log.info("is there anyway this could go to main.log and xyz.log?")
    parser.parse()

Is there a way I can set the Log file name from my main.py module and not from logger.py module?

Answer 1

Logging instances can have multiple file handlers. Use a function like this to just add another handler with the additional output path you want. Log messages will get sent to both (or all) text logs added to the instance. You can even configure the handlers to have different logging levels so you can filter messages to different logs for critical errors, info message, etc.

import logging
def add_handler(output_log_path, log):
    # Set up text logger and add it to logging instance
    file_logger = logging.FileHandler(output_log_path)
    file_logger.setLevel(logging.DEBUG)
    formatter = logging.Formatter('%(asctime)s | logger name: %(name)s | module: %(module)s | lineno: %(lineno)d | %(message)s')
    file_logger.setFormatter(formatter)
    log.addHandler(file_logger)
    return log

Answer 2

You're calling get_logger() first, so when you set the class attribute in FileName.file_name = "main.log" the get_logger function is already finished, and the logger is already defined to write in xyz.log; Changing the variable later won't change the logger anymore, since it is already defined.

To change the previously selected file, you'd have to retrieve the logger, remove the previous handler and add a new file handler. Another option is to set the variable before calling get_logger() so when you call it, the variable already has the correct value.