Using StreamWriter to implement a rolling log, and deleting from top

Question

My C# winforms 4.0 application has been using a thread-safe streamwriter to do internal, debug logging information. When my app opens, it deletes the file, and recreates it. When the app closes, it saves the file.

What I'd like to do is modify my application so that it does appending instead of replacing. This is a simple fix.

However, here's my question:

I'd like to keep my log file AROUND 10 megabytes maximum. My constraint would be simple. When you go to close the file, if the file is greater than 10 megabytes, trim out the first 10%.

Is there a 'better' way then doing the following:

1. Close the file
2. Check if the file is > 10 meg
3. If so, open the file
4. Parse the entire thing
5. Cull the first 10%
6. Write the file back out
7. Close

Edit: well, I ended up rolling my own (shown following) the suggestion to move overt to Log4Net is a good one, but the time it woudl take to learn the new library and move all my log statements (thousands) over isn't time effective for the small enhancement I was trying to make.

  private static void PerformFileTrim(string filename)
  {
     var FileSize = Convert.ToDecimal((new System.IO.FileInfo(filename)).Length);

     if (FileSize > 5000000)
     {
        var file = File.ReadAllLines(filename).ToList();
        var AmountToCull = (int)(file.Count * 0.33); 
        var trimmed = file.Skip(AmountToCull).ToList();
        File.WriteAllLines(filename, trimmed);
     }
  }

Answer 1

The solutions here did not really work for me. I took user3902302's answer, which again was based on bigtech's answer and wrote a complete class. Also, I am NOT using StreamWriter, you can change the one line (AppendAllText against the StreamWrite aequivalent).

There is little error handling (e. g. re-try access when it is failing, though the lock should catch all internal concurrent access).

This might be enough for some people who had to use a big solution like log4net or nlog before. (And log4net RollingAppender is not even thread-safe, this one is. :) )

public class RollingLogger
{
    readonly static string LOG_FILE = @"c:\temp\logfile.log";
    readonly static int MaxRolledLogCount = 3;
    readonly static int MaxLogSize = 1024; // 1 * 1024 * 1024; <- small value for testing that it works, you can try yourself, and then use a reasonable size, like 1M-10M

    public static void LogMessage(string msg)
    {
        lock (LOG_FILE) // lock is optional, but.. should this ever be called by multiple threads, it is safer
        {
            RollLogFile(LOG_FILE);
            File.AppendAllText(LOG_FILE, msg + Environment.NewLine, Encoding.UTF8);
        }
    }

    private static void RollLogFile(string logFilePath)
    {
        try
        {
            var length = new FileInfo(logFilePath).Length;

            if (length > MaxLogSize)
            {
                var path = Path.GetDirectoryName(logFilePath);
                var wildLogName = Path.GetFileNameWithoutExtension(logFilePath) + "*" + Path.GetExtension(logFilePath);
                var bareLogFilePath = Path.Combine(path, Path.GetFileNameWithoutExtension(logFilePath));
                string[] logFileList = Directory.GetFiles(path, wildLogName, SearchOption.TopDirectoryOnly);
                if (logFileList.Length > 0)
                {
                    // only take files like logfilename.log and logfilename.0.log, so there also can be a maximum of 10 additional rolled files (0..9)
                    var rolledLogFileList = logFileList.Where(fileName => fileName.Length == (logFilePath.Length + 2)).ToArray();
                    Array.Sort(rolledLogFileList, 0, rolledLogFileList.Length);
                    if (rolledLogFileList.Length >= MaxRolledLogCount)
                    {
                        File.Delete(rolledLogFileList[MaxRolledLogCount - 1]);
                        var list = rolledLogFileList.ToList();
                        list.RemoveAt(MaxRolledLogCount - 1);
                        rolledLogFileList = list.ToArray();
                    }
                    // move remaining rolled files
                    for (int i = rolledLogFileList.Length; i > 0; --i)
                        File.Move(rolledLogFileList[i - 1], bareLogFilePath + "." + i + Path.GetExtension(logFilePath));
                    var targetPath = bareLogFilePath + ".0" + Path.GetExtension(logFilePath);
                    // move original file
                    File.Move(logFilePath, targetPath);
                }
            }
        }
        catch (Exception ex)
        {
            System.Diagnostics.Debug.WriteLine(ex.ToString());
        }
    }
}

edit:
Since I just noticed that you asked a slightly different question: should your lines vary greatly in size, this would be a variation (, that in 90% of cases does not improve over yours, though, and might be very slightly faster, also introduced a new unhandled error (\n not being present)):

    private static void PerformFileTrim(string filename)
    {
        var fileSize = (new System.IO.FileInfo(filename)).Length;

        if (fileSize > 5000000)
        {
            var text = File.ReadAllText(filename);
            var amountToCull = (int)(text.Length * 0.33);
            amountToCull = text.IndexOf('\n', amountToCull);
            var trimmedText = text.Substring(amountToCull + 1);
            File.WriteAllText(filename, trimmedText);
        }
    }