Why is the stack overflow depending on how to access the array in Go?

Question

Consider the following Go program:

package main

func main() {
    var buffer [100000000]float64
    var i int
    for i = range buffer {
        buffer[i] = float64(i)
    }
}

With "go run test1.go", it works. (Unless you have too little RAM.)

Now, I expand this program trivially:

package main

func main() {
    var buffer [100000000]float64
    var i int
    var value float64
    for i, value = range buffer {
        value = value
        buffer[i] = float64(i)
    }
}

"go run test2.go" yields:

runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow

runtime stack:
runtime.throw(0x473350, 0xe)
        /usr/local/go/src/runtime/panic.go:527 +0x90
runtime.newstack()
        /usr/local/go/src/runtime/stack1.go:794 +0xb17
runtime.morestack()
        /usr/local/go/src/runtime/asm_amd64.s:330 +0x7f

goroutine 1 [stack growth]:
main.main()
        /home/bronger/src/go-test/test3.go:3 fp=0xc860075e50 sp=0xc860075e48
runtime.main()
        /usr/local/go/src/runtime/proc.go:111 +0x2b0 fp=0xc860075ea0 sp=0xc860075e50
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1696 +0x1 fp=0xc860075ea8 sp=0xc860075ea0
exit status 2

It seems to me that in test1.go, the heap was used, whereas in test2.go, the stack was used. Why?

Answer 1

According to the Go specification:

The range expression is evaluated once before beginning the loop, with one exception: if the range expression is an array or a pointer to an array and at most one iteration variable is present, only the range expression's length is evaluated

So in the first program only the length of buffer is evaluated and placed on the stack.

In the second program the whole buffer is placed on the stack before iterating over it. Because the size of the buffer is known during compilation the Go compiler places instruction at the beginning of the function to pre-allocate stack space. That is why panic trace points to the beginning of the function.

In both cases buffer is allocated on the heap. This can be confirmed by

$ go build -gcflags=-m
./main.go:4: moved to heap: buffer

Note that the program will behave similarly if you make buffer a global variable.

However if you change buffer to be a slice (buffer := make([]float64, 100000000)) the program will succeed in both cases. This happens because a slice is just a pointer to the actual array and it takes only a few bytes on the stack independently of the backing array size. So the simplest way to fix your second program is to make it iterate over slice instead of array:

....
for i, value = range buffer[:] {
    ....
}

Surprisingly if you try to create a bigger array [1000000000]float64 then the compiler will complain (stack frame too large (>2GB)). I think it should complain when compiling your second program as well instead of letting it panic. You might want to report this issue to http://github.com/golang/go/issues