Golang - Copy Exec output to Log

Question

I would like to redirect the output of a process to log in a timely manner. I can do it if I wait for the process to finish like this:

cmd := exec.Command("yes", "Go is awesome") // Prints "Go is awesome", forever 
out, err := cmd.CombinedOutput()
log.Printf("%s", out)

However, if the process takes a long time or doesn't finish this is less useful. I know I can write to stdout in real time like this:

cmd := exec.Command("yes")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Run()

This doesn't really help me though, because I am writing a service that isn't writing to a terminal. I'm looking for something that will let me do something like:

cmd := exec.Command("yes")
cmd.Stdout = log.Stdout
cmd.Stderr = log.Stdout
cmd.Run()

log doesn't give direct access to its writer so this is not possible. Surely I'm not the only with this problem, how is this typically done?

Answer 1

You should use a pipe here, for example:

stdout, err := cmd.StdoutPipe()
if err != nil {
    return 0, err
}

// start the command after having set up the pipe
if err := cmd.Start(); err != nil {
    return 0, err
}

// read command's stdout line by line
in := bufio.NewScanner(stdout)

for in.Scan() {
    log.Printf(in.Text()) // write each line to your log, or anything you need
}
if err := in.Err(); err != nil {
    log.Printf("error: %s", err)
}

I have only handled Stdout here, but it is possible to handle Stderr at the same time, for example by using a goroutine.

Answer 2

Both exec.Command and log.Logger are based on an io.Writer. You don't have access to those of the second, but you don't need to because either you didn't changed it, and you're using os.Stderr or you changed it and you have the io.Writer at hand when creating the logger.

So you just have to adapt you second example by using the right io.Writer…

Edit

After some though on it, I think you may be good to go by just embedding a log.Logger into a struct that will implement the io.Writer interface…

Example:

type LogWriter struct {
    logger *log.Logger
}

func NewLogWriter(l *log.Logger) *LogWriter {
    lw := &LogWriter{}
    lw.logger = l
    return lw
}

func (lw LogWriter) Write (p []byte) (n int, err error) {
    lw.logger.Println(p)
    return len(p), nil
}

And then pass it to your exec.Command output…

cmd := exec.Command(…)
cmd.Stdout = NewLogWriter(log.New(…))

You can use it for both standard or error output, or create new object for each one.

Edit2

As I'm going to use this trick for one of my next projects, I put it into a package on github. Feel free to give feedback, etc.