Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reading log files as they're updated in Go

Tags:

go

I'm trying to parse some log files as they're being written in Go but I'm not sure how I would accomplish this without rereading the file again and again while checking for changes.

I'd like to be able to read to EOF, wait until the next line is written and read to EOF again, etc. It feels a bit like how tail -f looks.

like image 469
Nick Avatar asked Apr 13 '12 05:04

Nick


People also ask

How do I read a Golang log file?

The basic idea would be to use Cmd from the "os/exec" package to follow the file. You could fork a process that was the equivalent of "tail --retry --follow=name prog. log", and then listen to it's Stdout using the Stdout reader on the the Cmd object.

How do I view file logs?

Locating Log Files. Most log files are located in the /var/log/ directory. Some applications such as httpd and samba have a directory within /var/log/ for their log files. You may notice multiple files in the log file directory with numbers after them.

How do you use logger in go?

By setting a logging level on a logger, you can log only the entries you need depending on your environment. By default, logrus will log anything that is Info or above (Warn, Error, Fatal, or Panic). package main import ( log "github.com/sirupsen/logrus" ) func main() { log. SetFormatter(&log.

Can log files be edited?

You can edit the file, but the entries added to the file while you edit it are lost, because the editor loads the log file into memory, and saves the file from memory to disk after editing. However, you can move the log file to another file with mv command, reload webserver, and then edit the other file.


1 Answers

I have written a Go package -- github.com/hpcloud/tail -- to do exactly this.

t, err := tail.TailFile("/var/log/nginx.log", tail.Config{Follow: true}) for line := range t.Lines {     fmt.Println(line.Text) } 

...

Quoting kostix's answer:

in real life files might be truncated, replaced or renamed (because that's what tools like logrotate are supposed to do).

If a file gets truncated, it will automatically be re-opened. To support re-opening renamed files (due to logrotate, etc.), you can set Config.ReOpen, viz.:

t, err := tail.TailFile("/var/log/nginx.log", tail.Config{     Follow: true,     ReOpen: true}) for line := range t.Lines {     fmt.Println(line.Text) } 

Config.ReOpen is analogous to tail -F (capital F):

 -F      The -F option implies the -f option, but tail will also check to see if the file being followed has been          renamed or rotated.  The file is closed and reopened when tail detects that the filename being read from          has a new inode number.  The -F option is ignored if reading from standard input rather than a file. 
like image 139
Sridhar Ratnakumar Avatar answered Sep 21 '22 23:09

Sridhar Ratnakumar