Go concurrency with for loop and anonymous function behaves unexpectedly

Question

I have already found out a way for the code to behave as I want, but I would like to understand why it behaves like this so that my understanding of Go concurrency improves.

I was testing out sync.WaitGroup to wait for some goroutines to finish because I plan on doing multiple uploads to Amazon S3 in this way.

This was the code I had originally:

func main() {

    var wg sync.WaitGroup

    for i := 1; i <= 5; i++ {
        wg.Add(1)
        go func() {
            fmt.Println(i)
            time.Sleep(time.Second * 1)
            wg.Done()
        }()
    }

    wg.Wait()
}

I was surprised to see that the output was: 6, 6, 6, 6, 6.

Instead of something like: 2, 4, 1, 5, 3.

Since the loop does not even go to 6, this made no sense to me. I later passed the i variable to the anonymous function as argument and then it behaved as I expected.

Why does this happen? I don't understand it.

Answer 1

This is covered in the faq: What happens with closures running as goroutines?

In this case, none of the goroutines get scheduled until the for loop completes. In order for the for loop to break i must not be less than or equal to 5, therefore it is 6 at that point. When the goroutines run, they each print the value of the single variable i which is captured in the closures.

When you pass i as an argument to the function, you copy the current value to a new variable, capturing the value at that moment.

Answer 2

To answer your question, you have to pass i into your func so that each routine would have its own copy of the value of i.

So your code should look like this

func main() {
    var wg sync.WaitGroup

    for i := 1; i <= 5; i++ {
        wg.Add(1)
        go func(i int) {
            fmt.Println(i)
            time.Sleep(time.Second * 1)
            wg.Done()
        }(i)
    }

    wg.Wait()
}

I wrote this utility function to help parallelize a group of functions:

import "sync"

// Parallelize parallelizes the function calls
func Parallelize(functions ...func()) {
    var waitGroup sync.WaitGroup
    waitGroup.Add(len(functions))

    defer waitGroup.Wait()

    for _, function := range functions {
        go func(copy func()) {
            defer waitGroup.Done()
            copy()
        }(function)
    }
}

So in your case, we could do this

func main() {
    functions := []func(){}
    for i := 1; i <= 5; i++ {
            function := func(i int) func() {
                return func() {
                        fmt.Println(i)
                }
            }(i)

        functions = append(functions, function)
    }

    Parallelize(functions...)

    fmt.Println("Done")
}

If you wanted to use the Parallelize function, you can find it here https://github.com/shomali11/util