Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Logback: SizeAndTimeBasedRollingPolicy applies totalSizeCap to each day in maxHistory

Logback version 1.2.3

I want to use a SizeAndTimeBasedRollingPolicy in our logback configuration file (logback.xml), but at this time the SizeAndTimeBasedRollingPolicy doesn't function as expected. (https://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy)

Ideally I want to keep logs dating back no later than ex. 90 days, each file no larger than 100MB and a total archive size of ex. 10GB in total.

As it currently stands, the totalSizeCap is applied to each entry within the MaxHistory range. Ex.

<appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>mylog.txt</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
      <!-- rollover daily -->
      <fileNamePattern>mylog-%d{yyyy-MM-dd}.%i.txt</fileNamePattern>
       <!-- each file should be at most 100MB, keep 60 days worth of history, but at most 20GB -->
       <maxFileSize>100MB</maxFileSize>    
       <maxHistory>60</maxHistory>
       <totalSizeCap>1GB</totalSizeCap>
    </rollingPolicy>
    <encoder>
      <pattern>%msg%n</pattern>
    </encoder>
</appender>

The above XML configuration will create logs spanning over 60 days, applying a totalSizeCap of 1GB per day. This will lead to a total archive size of 60GB instead of the expected 1GB. If the totalSizeCap is reached during the day the day's logs will start to rollover by deleting the day's oldest files, this will create gaps within the log history, which we don't want. A work-around for this error was to use a yearly rollover, instead of daily or monthly rollover, unfortunately yearly rollover doesn't work when using a SizeAndTimeBasedRollingPolicy.

Does anyone know of this problem, has this been fixed, or am I doing something wrong in my configuration?

like image 670
Iggydv Avatar asked Dec 11 '17 12:12

Iggydv


People also ask

What is sizeandtimebasedrollingpolicy?

Size and time based rolling policy Sometimes you may wish to archive files essentially by date but at the same time limit the size of each log file, in particular if post-processing tools impose size limits on the log files. In order to address this requirement, logback ships with SizeAndTimeBasedRollingPolicy.

What is the maximum size of logs for Logback?

The given logback configuration creates daily rolled-over logs with max log size of 10 MB, keeping 30 days worth of history, but at most 10 GB of total archived logs. Older logs start getting deleted as soon as the size limit is breached. 2.3. SizeBasedTriggeringPolicy

What is Logback RollingFileAppender?

Logback RollingFileAppender appends log events into a file with the capability to rollover ( archive the current log file and resume logging in a new file) based on a particular schedule, such as daily, weekly, monthly or based on log file size. For quick reference, this is configuration file we will be discussing further in the post.

What version of Logback does totalsizecap work?

I have checked, totalSizeCap works in Logback 1.1.8-SNAPSHOT. Well even the question is answered, I wanted to post the work around that we made until the bug is fixed in 1.1.8.


1 Answers

It seems that you have found a bug!

There should be no gaps in the logs. When totalSizeCap is reached, the oldest log file should be deleted. When maxHistory reached, the oldest log file should be deleted.

Unfortunately it seems that there is a bug in logback which causes gaps in the logging because not the oldest files are deleted. See demonstration here: https://github.com/riskop/slf4j_logback_SizeAndTimeBasedRollingPolicy_problem

I've opened an issue: https://jira.qos.ch/browse/LOGBACK-1361

According to Gülcü it will be fixed in logback classic 1.3.0.

Note that you can "vote" the issue on logback's Jira!

like image 63
riskop Avatar answered Sep 22 '22 01:09

riskop