Cannot understand 5.6.1. Caveat: Capturing Iteration Variables

Question

I'm learning Go and cannot understand

 var rmdirs []func()

 for _, dir := range tempDirs() {
     os.MkdirAll(dir, 0755)
     rmdirs = append(rmdirs, func() {
         os.RemoveAll(dir) // NOTE: incorrect!
     })
 }

I've read the explain in the book several times and still cannot figure it out why incorrect?

I remember that in go arguments are passed by value, so every loop dir is different value, why incorrect?

Answer 1

Your intuition is correct: go reuses the same address for the iteration values, so there is no guarantee that the value pointed to by dir when the anonymous function appended to rmdirs has the same value it did when the function was created and dir was first captured. The exact wording in the specs is:

The iteration variables may be declared by the "range" clause using a form of short variable declaration (:=). In this case their types are set to the types of the respective iteration values and their scope is the block of the "for" statement; they are re-used in each iteration. If the iteration variables are declared outside the "for" statement, after execution their values will be those of the last iteration.

(Emphasis mine). To further demonstrate, here is a simplified version of what your code is trying to do:

var rmdirs []func()
tempDirs := []string{"one", "two", "three", "four"}

for _, dir := range tempDirs {
    fmt.Printf("dir=%v, *dir=%p\n", dir, &dir)
    rmdirs = append(rmdirs, func() {
        fmt.Printf("dir=%v, *dir=%p\n", dir, &dir)
    })
}

fmt.Println("---")

for _, f := range rmdirs {
    f()
}

When run, this produces the following output:

dir=one, *dir=0x40e128
dir=two, *dir=0x40e128
dir=three, *dir=0x40e128
dir=four, *dir=0x40e128
---
dir=four, *dir=0x40e128
dir=four, *dir=0x40e128
dir=four, *dir=0x40e128
dir=four, *dir=0x40e128

Playground link: https://play.golang.org/p/_rS8Eq9qShM

As you can see, the same address is reused each iteration through the loop. Each iteration of the anonymous function is looking at the same address, so they all print the same value.

The correct way to handle situations like this, as mentioned in the book you reference, is by defining a new variable within the loop, copying the iteration value to that, and passing that to the anonymous function, like so:

var rmdirs []func()
tempDirs := []string{"one", "two", "three", "four"}

for _, d := range tempDirs {
    dir := d
    fmt.Printf("dir=%v, *dir=%p\n", dir, &dir)
    rmdirs = append(rmdirs, func() {
        fmt.Printf("dir=%v, *dir=%p\n", dir, &dir)
    })
}

fmt.Println("---")

for _, f := range rmdirs {
    f()
}

This produces output that you would expect:

dir=one, *dir=0x40e128
dir=two, *dir=0x40e150
dir=three, *dir=0x40e168
dir=four, *dir=0x40e180
---
dir=one, *dir=0x40e128
dir=two, *dir=0x40e150
dir=three, *dir=0x40e168
dir=four, *dir=0x40e180

Playground link: https://play.golang.org/p/Ao6fC9i2DsG