Why is this program not performing better with goroutines?

Question

I am learning Go programming language. Please consider the following program,

package main

import (
    "fmt"
    "bytes"
    "os"
    "os/exec"
    "path/filepath"
    "sync"
)

func grep(file string) {
    defer wg.Done()

    cmd := exec.Command("grep", "-H", "--color=always", "add", file)
    var out bytes.Buffer
    cmd.Stdout = &out
    cmd.Run()
    fmt.Printf("%s\n", out.String())
}

func walkFn(path string, info os.FileInfo, err error) error {
    if !info.IsDir() {
        wg.Add(1)
        go grep (path)
    }
    return nil
}

var wg sync.WaitGroup

func main() {
    filepath.Walk("/tmp/", walkFn)
    wg.Wait()
}

This program walks all the files in the /tmp directory, and does a grep on each file in a goroutine. So this will spawn n goroutines where n is the number of files present in the /tmp directory. Main waits till all goroutines finishes the work.

Interestingly, this program take same time to execute with and without goroutines. Try running go grep (path, c) and grep (path, c) (you need to comment channel stuff when doing this).

I was expecting goroutine version to run faster as multiple grep runs concurrently. But it executes almost in equal time. I am wondering why this happens?

Answer 1

Try using more cores. Also, use a better root directory for comparative purposes, like the Go directory. An SSD makes a big difference too. For example,

func main() {
    runtime.GOMAXPROCS(runtime.NumCPU())
    goroot := "/home/peter/go/"
    filepath.Walk(goroot, walkFn)
    wg.Wait()
    fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))
}

GOMAXPROCS: 1
real    0m10.137s
user    0m2.628s
sys     0m6.472s

GOMAXPROCS: 4
real    0m3.284s
user    0m2.492s
sys     0m5.116s

Answer 2

Your program's performance is bound to the speed of the disk (or ram, if /tmp is a ram disk): the computation is I/O bound. No matter how many goroutines run in parallel it can't read faster than that.