How can I log in golang to a file with log rotation?

Question

I am trying to write a web application which will run on a remote server. I need to log to capture errors/debug/audit. I find that multiple logging packages are available for golang including the standard "log" package. However, I need to fulfill three requirements:

1. The log files need to be rotated
2. It applies to the included packages which use "log"
3. It needs to be cross-platform. Dev environment is Linux and needs to be deployed on Windows.

Answer 1

Though @Crast has given a very good answer, I want to also bring to the notice - lumberjack logger by Nate Finch which I ended up using.

Here is how to use it:

1. First, clone the lumberjack repository OR get it somehow.
2. Run the go install command on the folder.
3. Now import go's "log" package and "lumberjack package".

import ( "log" "github.com/natefinch/lumberjack" )

4. Now use it in your code like this:

Outside of main, declare your log variable.

var errLog *log.Logger 

Inside main:

e, err := os.OpenFile("./foo.log", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0666)  if err != nil {     fmt.Printf("error opening file: %v", err)     os.Exit(1) } errLog = log.New(e, "", log.Ldate|log.Ltime) errLog.SetOutput(&lumberjack.Logger{     Filename:   "./foo.log",     MaxSize:    1,  // megabytes after which new file is created     MaxBackups: 3,  // number of backups     MaxAge:     28, //days }) 

Now as soon as the file size get 1MB, a new file is created to keep the previous logs with the current timestamps, and the new logs will continue to log into foo.log file. Also, I have created the file using os.OpenFile but you may not need it as lumberjack internally does it, but I preferred it that way. Thanks, hope it helps. Once again thanks to @Crast and NateFinch.

Answer 2

The best way to fulfill all your three requirements instead of creating an alternate logger struct, if you were satisfied using the base-level log.Log, is instead to set the output of the logger to your own io.Writer instance.

So basically what I'm going to do here is show an example where I create my own io.Writer:

import (     "os"     "sync"     "time" )  type RotateWriter struct {     lock     sync.Mutex     filename string // should be set to the actual filename     fp       *os.File }  // Make a new RotateWriter. Return nil if error occurs during setup. func New(filename string) *RotateWriter {     w := &RotateWriter{filename: filename}     err := w.Rotate()     if err != nil {         return nil     }     return w }  // Write satisfies the io.Writer interface. func (w *RotateWriter) Write(output []byte) (int, error) {     w.lock.Lock()     defer w.lock.Unlock()     return w.fp.Write(output) }  // Perform the actual act of rotating and reopening file. func (w *RotateWriter) Rotate() (err error) {     w.lock.Lock()     defer w.lock.Unlock()      // Close existing file if open     if w.fp != nil {         err = w.fp.Close()         w.fp = nil         if err != nil {             return         }     }     // Rename dest file if it already exists     _, err = os.Stat(w.filename)     if err == nil {         err = os.Rename(w.filename, w.filename+"."+time.Now().Format(time.RFC3339))         if err != nil {             return         }     }      // Create a file.     w.fp, err = os.Create(w.filename)     return } 

You then create a RotateWriter and use log.SetOutput to set this writer (if other packages are using the standard logger instance) or alternately create your own instances using log.New to pass around.

I haven't solved the situation of when to call Rotate, I'll leave that to you to decide. It'd be fairly simple to trigger it based on time, or alternately do so after some amount of writes or some amount of bytes.