Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python logging and rotating files

Tags:

python

logging

I have a python program that is writing to a log file that is being rotated by Linux's logrotate command. When this happens I need to signal my program to stop writing to the old file and start writing to the new one. I can handle the signal but how do I tell python to write to the new file?

I am opening the file like this:

logging.basicConfig(format='%(asctime)s:%(filename)s:%(levelname)s:%(message)s',filename=log_file, level=logging.INFO)

and writing to it like this:

logging.log(level,"%s" % (msg))

The logging modules look very powerful but also overwhelming. Thanks.

like image 924
fredsnertz Avatar asked Feb 02 '12 03:02

fredsnertz


People also ask

What is TimedRotatingFileHandler in Python?

TimedRotatingFileHandler. The TimedRotatingFileHandler class, located in the logging. handlers module, supports rotation of disk log files at certain timed intervals.

What is a rotating log file?

In information technology, log rotation is an automated process used in system administration in which log files are compressed, moved (archived), renamed or deleted once they are too old or too big (there can be other metrics that can apply here).


3 Answers

Don't use logging.basicConfig, use WatchedFileHandler. Here's how to use it.

import time
import logging
import logging.handlers

def log_setup():
    log_handler = logging.handlers.WatchedFileHandler('my.log')
    formatter = logging.Formatter(
        '%(asctime)s program_name [%(process)d]: %(message)s',
        '%b %d %H:%M:%S')
    formatter.converter = time.gmtime  # if you want UTC time
    log_handler.setFormatter(formatter)
    logger = logging.getLogger()
    logger.addHandler(log_handler)
    logger.setLevel(logging.DEBUG)

log_setup()
logging.info('Hello, World!')
import os
os.rename('my.log', 'my.log-old')
logging.info('Hello, New World!')
like image 167
proski Avatar answered Oct 21 '22 05:10

proski


You may want to look at WatchedFileHandler to implement this, or as an alternative, implement log rotation with RotatingFileHandler, both of which are in the logging.handlers module.

like image 29
Andrew Walker Avatar answered Oct 21 '22 06:10

Andrew Walker


from logging import handlers

handler = handlers.TimedRotatingFileHandler(filename, when=LOG_ROTATE)

handler.setFormatter(logging.Formatter(log_format, datefmt="%d-%m-%Y %H:%M:%S"))

#LOG_ROTATE = midnight    
#set your log format

This should help you in handling rotating log

like image 10
avasal Avatar answered Oct 21 '22 04:10

avasal