Rolling file appender based on size limit and backing up with a timestamp

Question

Is there a way I can set up log4net config such a way that if log file's size exceeds a certain limit existing log file should be renamed with a timestamp?

Requirement:

E.g. When size exceeds 1MB, then my logs should look like

Log.txt
Log.12-07-2018.txt
Log.11-07-2018.txt

etc, where Log.txt is the current/latest log.

Note:

1. When rolling happens twice on the same day then log4net should append it to the file that already has the day's timestamp. For e.g. if roll on happens twice on 12-07-2018 and file size limit is 1MB it would be OK to back up to Log.12-07-2018.txt file result in 2MB file size.

What I tried:

This is the log4net config I'm using:

<log4net>
  <appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender" >
    <file value="${LOCALAPPDATA}\foobar\Log.txt" />
    <encoding value="utf-8" />
    <appendToFile value="true" />
    <rollingStyle value="Size" />
    <maximumFileSize value="1MB" />
    <maxSizeRollBackups value="1000" />
    <preserveLogFileNameExtension value="true" />
    <datePattern value=".dd-MM-yyyy" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date %level [%thread] %type.%method - %message%n" />
    </layout>
  </appender>
  <root>
    <level value="All" />
    <!-- If the following line is not included the log file will not be created even if log4net is configured with this file. -->
    <appender-ref ref="RollingFileAppender" />
  </root>
</log4net>

What I get:

Log.txt
Log.1.txt
Log.2.txt

---

I want the renaming part to be based on timestamp and backup logic based on size. When I try:

    <rollingStyle value="Date" />
    <datePattern value=".dd-MM-yyyy" />

The naming of backup log file works, but then rolling happens based on date, not size.

Answer 1

It can be achieved, but not via configuration only.
A custom RollingFileAppender must be created and be referenced in the Log4net configuration as shown here below, where PFX.RollingFileAppender is loaded from assembly PFX.Lib.

The setting staticLogFileName ensures that the most recent file is always named Log.txt (as configured). We (re)use the datePattern option to configure the date part in the log file name.
(For performance reasons, countDirection gets set to value 0 to reduce the number of rollovers; the most recent/last backup file will be the one with the highest number.)

<log4net>
    <appender name="RollingFileAppender" type="PFX.CustomRollingFileAppender, PFX.Lib" >
        <datePattern value="yyyy-MM-dd" />
        <staticLogFileName value="true" />
        <countDirection value="0" />
        <file value="${LOCALAPPDATA}\foobar\Log.txt" />
        <encoding value="utf-8" />
        <appendToFile value="true" />
        <rollingStyle value="Size" />
        <maximumFileSize value="1MB" />
        <maxSizeRollBackups value="1000" />
        <preserveLogFileNameExtension value="true" />            
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%date %level [%thread] %type.%method - %message%n" />
        </layout>
    </appender>
    <root>
        <level value="All" />    
        <appender-ref ref="RollingFileAppender" />
    </root>
</log4net>

The most challenging part in the requirements is the exclusing of the index/counter suffix in the file name; eg. Log-2018-07-16.1.txt as Log4net internally tracks and uses this counter number to take care of eg. the backup file deletion. Because of this, we'll have to manage the cleanup ourselves.

Log4net's implementation of the RollingFileAppender is not very open; there are very few methods to override and there is no access to the current counter number tracking.
If you can't or don't want to include and modify the full source code of the original RollingFileAppender in your own project, you can inherit from the original one in order to make as few as possible changes, as shown below.

If a rollup based on file size occurs we check whether todays log file (matching todays date; see requirement) already exists. If so, no rollup occurs; only the content from Log.txt gets moved to todays log file.
If todays log file does not exist, an out-of-the-box rollover occurs. This creates a file with an index/counter suffix in its name; immediately afterwards this file gets renamed to match with the configured date pattern. Because a new file got created, any depricated log files (more than the configured treshold) get deleted.

(For brevity, the code below contains no exception handling.)

using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using log4net.Appender;
using log4net.Util;

namespace PFX
{
    public class CustomRollingFileAppender : RollingFileAppender
    {
        private String _baseFileExtension;
        private String _baseFileNameWithoutExtension;        
        private String _fileDeletePattern;
        private String _folder;        
        private String _backupSearchPattern;

        public CustomRollingFileAppender()
        {}

        public override void ActivateOptions()
        {
            base.ActivateOptions();

            this._baseFileNameWithoutExtension = Path.GetFileNameWithoutExtension(this.File);
            this._baseFileExtension = Path.GetExtension(this.File);
            this._folder = Path.GetDirectoryName(this.File);            
            this._fileDeletePattern = $"{this._baseFileNameWithoutExtension}*{this._baseFileExtension}";
            this._backupSearchPattern = $"{this._baseFileNameWithoutExtension}.*{this._baseFileExtension}";            
        }               

        protected override void AdjustFileBeforeAppend()
        {   
            if ((RollingMode.Size == this.RollingStyle) 
                && (this.File != null)
                && (((CountingQuietTextWriter)base.QuietWriter).Count >= this.MaxFileSize)
                )
            {                   
                DateTime now = DateTime.Now;
                String todayFileSuffix = now.ToString(this.DatePattern, CultureInfo.InvariantCulture);
                String todayFileName = $"{this._baseFileNameWithoutExtension}{todayFileSuffix}{this._baseFileExtension}";
                String todayFile = Path.Combine(this._folder, todayFileName);

                if (base.FileExists(todayFile))
                {                    
                    /* Todays logfile already exist; append content to todays file. */
                    base.CloseFile(); // Close current file in order to allow reading todays file.
                    this.moveContentToTodaysFile(todayFile);
                    // Delete and open todays file for a fresh start.
                    base.DeleteFile(this.File);
                    base.OpenFile(this.File, false);
                }
                else
                {
                    /* Do a roll-over. */

                    base.RollOverSize();

                    using (base.SecurityContext.Impersonate(this))
                    {   
                        this.deleteDepricatedBackupFiles();
                        this.renameBackupFiles(todayFile);
                    }
                }
            }
            else
            {
                base.AdjustFileBeforeAppend();
            }
        }

        // Moves the content from the current log file to todays file.
        private void moveContentToTodaysFile(String todayFile)
        {
            using (FileStream logFile = new FileStream(this.File, FileMode.Open, FileAccess.Read, FileShare.Read))
            using (StreamReader reader = new StreamReader(logFile))
            using (FileStream backupFile = new FileStream(todayFile, FileMode.Append, FileAccess.Write, FileShare.ReadWrite))
            using (StreamWriter writer = new StreamWriter(backupFile))
            {
                const Int32 BUFFER_SIZE = 1024;
                Char[] buffer = new Char[BUFFER_SIZE];

                while (true)
                {
                    Int32 nrOfCharsRead = reader.Read(buffer, 0, BUFFER_SIZE);
                    if (nrOfCharsRead <= 0) { break; }

                    writer.Write(buffer, 0, nrOfCharsRead);
                }
            }
        }

        // Rename backup files according to the configured date pattern, removing the counter/index suffix.
        private void renameBackupFiles(String todayFile)
        {
            IEnumerable<String> backupFiles = Directory.EnumerateFiles(this._folder, this._backupSearchPattern, SearchOption.TopDirectoryOnly);
            foreach (String backupFile in backupFiles)
            {   
                base.RollFile(backupFile, todayFile);
            }
        }

        // Keep the number of allowed backup files and delete all others.
        private void deleteDepricatedBackupFiles()
        {
            DirectoryInfo folder = new DirectoryInfo(this._folder);    

            IEnumerable<FileInfo> filesToDelete = 
                folder
                    .EnumerateFiles(this._fileDeletePattern, SearchOption.TopDirectoryOnly)
                    .OrderByDescending(o => o.LastWriteTime)
                    .Skip(this.MaxSizeRollBackups + 1)
                    ;

            foreach (FileSystemInfo fileToDelete in filesToDelete)
            {
                base.DeleteFile(fileToDelete.FullName);
            }
        }        
    }
}