logging - rewrite if size gets too big?

Question

I have a daemon and it logs its operations:

LOG_FILE = '%s/%s.log' % (LOG_DIR, machine_name)
logging.basicConfig(filename=LOG_FILE,
                    level=logging.DEBUG,
                    format='%(asctime)s,%(msecs)05.1f '
                           '(%(funcName)s) %(message)s',
                    datefmt='%H:%M:%S')

After a while, the log size got really big and there was no space left on the device.

So I had to manually remove the logs and it was fine.

What's the best way to keep the log size not to go over certain limit? Can it be done via logging module or do i need to write external script to just remove the logs every once in a while?

Answer 1

You could use RotatingFileHandler to do the log rotation for you. There's an example in Logging Cookbook at Python docs:

import logging
import logging.handlers

LOG_FILENAME = 'logging_rotatingfile_example.out'

# Set up a specific logger with our desired output level
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)

# Add the log message handler to the logger
handler = logging.handlers.RotatingFileHandler(
              LOG_FILENAME, maxBytes=20, backupCount=5)

my_logger.addHandler(handler)

# Log some messages
for i in range(20):
    my_logger.debug('i = %d' % i)