Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get the realtime output for a shell command in golang?

Tags:

shell

go

I am trying to call shell command with os/exec in golang, that command will take some time, so I would like to retrieve the reatime output and print the processed output (a progressing ratio number).

package main

import (
    "bufio"
    "fmt"
    "io"
    "os"
    "os/exec"
    "strings"
)

func main() {
    cmdName := "ffmpeg -i t.webm  -acodec aac -vcodec libx264  cmd1.mp4"
    cmdArgs := strings.Fields(cmdName)

    cmd := exec.Command(cmdArgs[0], cmdArgs[1:len(cmdArgs)]...)
    stdout, _ := cmd.StdoutPipe()
    cmd.Start()
    go print(stdout)
    cmd.Wait()
}

// to print the processed information when stdout gets a new line
func print(stdout io.ReadCloser) {
    r := bufio.NewReader(stdout)
    line, _, err := r.ReadLine()
    fmt.Println("line: %s err %s", line, err)
}

I want to have a function where can update the screen when the command print something,

The ffmpeg command output is as follows:

frame=  101 fps=0.0 q=28.0 size=      91kB time=00:00:04.13 bitrate= 181.2kbits/
frame=  169 fps=168 q=28.0 size=     227kB time=00:00:06.82 bitrate= 272.6kbits/
frame=  231 fps=153 q=28.0 size=     348kB time=00:00:09.31 bitrate= 306.3kbits/
frame=  282 fps=140 q=28.0 size=     499kB time=00:00:11.33 bitrate= 360.8kbits/

in fact, the above 4 line is the last line of ffmpeg command output which keeps changing, I want to print that change out, like

18%
44%
69%
100%

how could I achieve this?

like image 595
seaguest Avatar asked May 07 '16 17:05

seaguest


People also ask

How do I find the shell output?

Get output from shell command using subprocess Launch the shell command that we want to execute using subprocess. Popen function. The arguments to this command is the shell command as a list and specify output and error. The output from subprocess.

How do you capture output of a command in a variable in a shell script?

Here are the different ways to store the output of a command in shell script. You can also use these commands on terminal to store command outputs in shell variables. variable_name=$(command) variable_name=$(command [option ...] arg1 arg2 ...) OR variable_name=`command` variable_name=`command [option ...]

What is exec command?

The exec command is a powerful tool for manipulating file-descriptors (FD), creating output and error logging within scripts with a minimal change. In Linux, by default, file descriptor 0 is stdin (the standard input), 1 is stdout (the standard output), and 2 is stderr (the standard error).


Video Answer


2 Answers

Looks like ffmpeg sends all diagnostic messages (the "console output") to stderr instead of stdout. Below code works for me.

package main

import (
    "bufio"
    "fmt"
    "os/exec"
    "strings"
)

func main() {
    args := "-i test.mp4 -acodec copy -vcodec copy -f flv rtmp://aaa/bbb"
    cmd := exec.Command("ffmpeg", strings.Split(args, " ")...)

    stderr, _ := cmd.StderrPipe()
    cmd.Start()

    scanner := bufio.NewScanner(stderr)
    scanner.Split(bufio.ScanWords)
    for scanner.Scan() {
        m := scanner.Text()
        fmt.Println(m)
    }
    cmd.Wait()
}

The version of ffmpeg is detailed as below.

ffmpeg version 3.0.2 Copyright (c) 2000-2016 the FFmpeg developers
built with Apple LLVM version 7.3.0 (clang-703.0.29)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.0.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-vda
libavutil      55. 17.103 / 55. 17.103
libavcodec     57. 24.102 / 57. 24.102
libavformat    57. 25.100 / 57. 25.100
libavdevice    57.  0.101 / 57.  0.101
libavfilter     6. 31.100 /  6. 31.100
libavresample   3.  0.  0 /  3.  0.  0
libswscale      4.  0.100 /  4.  0.100
libswresample   2.  0.101 /  2.  0.101
libpostproc    54.  0.100 / 54.  0.100
like image 175
Browny Lin Avatar answered Oct 27 '22 18:10

Browny Lin


I do find icza's solution that he mentioned in that post is quite useful, however it didn't't solve my problem.

I did a little test as following:

1, I write a script which print some info every second for ten times, here is the script.sh

#!/bin/bash

for i in {1..10}
do
    echo "step " $i
    sleep 1s
done

2, read the stdout and extract the needed information from stdout and do some process to get the expected format, here is the code: package main

import (
    "fmt"
    "os/exec"
    "regexp"
    "strconv"
    "strings"
)

func getRatio(text string) float32 {
    re1, _ := regexp.Compile(`step[\s]+(\d+)`)
    result := re1.FindStringSubmatch(text)
    val, _ := strconv.Atoi(result[1])
    return float32(val) / 10
}

func main() {
    cmdName := "ffmpeg -i t.webm  -acodec aac -vcodec libx264  cmd1.mp4"
    //cmdName := "bash ./script.sh"
    cmdArgs := strings.Fields(cmdName)

    cmd := exec.Command(cmdArgs[0], cmdArgs[1:len(cmdArgs)]...)
    stdout, _ := cmd.StdoutPipe()
    cmd.Start()

    oneByte := make([]byte, 10)
    for {
        _, err := stdout.Read(oneByte)
        if err != nil {
            break
        }
        progressingRatio := getRatio(string(oneByte))
        fmt.Printf("progressing  ratio %v \n", progressingRatio)
    }
}

This does work for my script.sh test, but for the ffmpeg command it doesn't work, in ffmpeg's case, nothing get printed and the process get finished (not stuck), I guess the way of writing data to stdout for ffmpeg is a little special (maybe no newline character at all, and I tried icza's solution, but it still doesn't work).

like image 32
seaguest Avatar answered Oct 27 '22 17:10

seaguest