Logback Keep n Days Logs With Size Based Archiving

Question

I'm trying to configure logs rolling policy for a java web application. Here is what I have so far

        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOG_HOME}/appLog.%d{yyyy-MM-dd HH}.%i.log.gz
            </fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy
                class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>500MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
            <maxHistory>7</maxHistory>
            <cleanHistoryOnStart>true</cleanHistoryOnStart>
        </rollingPolicy>

What I'm trying to achieve here is to keep all the logs for the past 7 days, but compress log files whenever they reach 500MB.

Logback configuration shown above keeps only last 7 files, so if there are lot of logs, I can have 7 files only for the last day, for example.

How can I configure logback in this case to keep all logs for the past 7 days? Any help is really appreciated.

Answer 1

What I'm trying to achieve here is to keep all the logs for the past 7 days,

Here you specify a pattern that considers a rolling policy until the hour granularity :

 <fileNamePattern>${LOG_HOME}/appLog.%d{yyyy-MM-dd HH}.%i.log.gz

The fileNamePattern is much more than a file name pattern.
It indicates also the rollover frequency.

The mandatory fileNamePattern property defines the name of the rolled-over (archived) log files. Its value should consist of the name of the file, plus a suitably placed %d conversion specifier. The %d conversion specifier may contain a date-and-time pattern as specified by the java.text.SimpleDateFormat class. If the date-and-time pattern is omitted, then the default pattern yyyy-MM-dd is assumed. The rollover period is inferred from the value of fileNamePattern.

By specifying this date pattern : %d{yyyy-MM-dd HH}, for each day of the year, at each time you log something at a new hour, the actual logs are archived and a new log file is created for the new hour.

For example : all what you log between the 01/03/17 09:00 and the 01/03/17 09:59are logged in the current log file.
As soon as 01/03/17 10:00, the first log you will do, will result to a log rotation (archive actual logs in a gz file with the filename pattern specified and clear the current log file).

Besides, as you specify <maxHistory>7</maxHistory>, it will archive only 7 distinct hours.
So if your application logs at each hour something, you will archive 7 hours of your application logs and not 7 days as you wish.

If you want to have logs history for 7 days, just remove the HH in the pattern :

 <fileNamePattern>${LOG_HOME}/appLog.%d{yyyy-MM-dd}.%i.log.gz

but compress log files whenever they reach 500MB.

When in fileNamePattern you specify a compression extension as a suffix of the file as gz, zip, etc ... the compression is automatic :

Note that file compression is also specified via this property. For example, fileNamePattern set to MyLogFile%i.log.zip means that archived files must be compressed using the zip format; gz format is also supported.

And anyway, Logback doesn't provide a way to compress archive only when a specific condition is encountered.
Logs should be compressed or should not.

When you declare this property :

 <maxFileSize>500MB</maxFileSize>

You specify the max file size by archive.
If you don't need to set a limit in the archive size, don't specify it and leave the default value (10MB).

You can retrieve all these information in the logback appenders documentation :

Answer 2

<maxHistory>168</maxHistory> would work in such case(24*7)