Limit Python log file

Question

I'm using Python3 (Ubuntu 14.04) to write out a log file. I want to limit the size of the log file and so chose RotatingFileHandler. Additionally, I don't want to rotate the log files (.log becomes .log.1 and so on) - I just want to limit the size of the log file.

I've tried using RotatingFileHandler like this:

filehandler = logging.handlers.RotatingFileHandler( "app.log", "a", 1000, 0, None, True )
logging.basicConfig( format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s", level = logging.DEBUG, handlers = [ filehandler ] )

What I'm finding is the log file continues to grow and is never truncated or "wrapped" at the limit.

I would have thought setting backupCount = 0 would cause the log file size to be checked before writing and if there's room for the log message, write it, otherwise empty the file and write out the message.

Any ideas?

Thanks in advance,

Bernmeister.

Answer 1

If you have a look at the source code of the RotatingFileHandler, you will see that a backupCount of 0 just closes and then reopens the log's stream. Here is a modified version that does what you want, discarding the old file:

import os
import logging
import logging.handlers


class TruncatedFileHandler(logging.handlers.RotatingFileHandler):
    def __init__(self, filename, mode='a', maxBytes=0, encoding=None, delay=0):
        super(TruncatedFileHandler, self).__init__(
            filename, mode, maxBytes, 0, encoding, delay)

    def doRollover(self):
        """Truncate the file"""
        if self.stream:
            self.stream.close()
        dfn = self.baseFilename + ".1"
        if os.path.exists(dfn):
            os.remove(dfn)
        os.rename(self.baseFilename, dfn)
        os.remove(dfn)
        self.mode = 'w'
        self.stream = self._open()

Here is how to use it:

filehandler = TruncatedFileHandler("app.log", "w", 1000)
logging.basicConfig(
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
    level=logging.DEBUG, handlers=[filehandler])

logger = logging.getLogger('spam_application')

for i in range(100):
    logger.debug("a log line, number: %s", i)

And proof that it works

❯❯❯ python3.4 main.py
❯❯❯ cat app.log*
2014-06-11 11:50:44,871 - spam_application - DEBUG - a log line, number: 91
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 92
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 93
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 94
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 95
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 96
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 97
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 98
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 99