Why does allocating a large element on the stack not fail in this specific case?

Question

When allocating a int as well as a large array on the stack in C, the program executes without error. If I however, initialize the variable on the stack beforehand, it crashes with a segfault (probably because the stack size was exceeded by the large array). If initializing the variable after declaring the array this would make sense to me. What causes this behavior, memory wise?

I was under the impression, that by simply declaring a variable on the stack, the needed space would be allocated, leading to an immediate crash when allocating very large datatypes.

My suspicion is that it has something to do with the compiler optimizing it away, but it does not make sense, considering I am not changing foo in the second example either.

I am using gcc 7.2.0 to compile, without any flags set. Executed on Ubuntu 17.10.

This runs without errors:

int main(){
  int i;
  unsigned char foo [1024*1024*1024];
  return 0;
}

while this crashes immediately:

int main(){
  int i = 0;
  unsigned char foo [1024*1024*1024];
  return 0;
}

Can somebody give me some insight what is happening here?

Answer 1

Note: What follows are implementation details. The C standard does not cover this.

The crash is not caused by allocating space. The crash is caused by writing to pages which are not writable, or reading from pages which are not readable.

You can see that a declaration doesn't actually need to read or write any memory, not necessarily:

int i;

But if it is initialized, you have to write the value:

int i = 0;

This triggers the crash. Note that the exact behavior will depend on the compiler you use and the optimization settings you have. Different compilers will allocate variables in different ways, and an optimizing compiler will normally remove both i and foo from the function entirely, since they aren't needed. Some compilers will also initialize variables to garbage values under certain configurations, to aid with debugging.

Allocating stack space just involves changing the stack pointer, which is a register. If you allocate too much stack space, the stack pointer will point to an invalid region of memory, and the program will segfault when it tries to read or write to those addresses. Most operating systems have “guard pages” so valid memory will not be placed next to the stack, ensuring that the program successfully crashes in most scenarios.

Here is some output from Godbolt:

main:
  push rbp
  mov rbp, rsp
  sub rsp, 1073741720        ; allocate space for locals
  mov DWORD PTR [rbp-4], 0   ; initialize i = 0
  mov eax, 0                 ; return value = 0
  leave
  ret

Note that this version does not crash, because i is placed at the top of the stack (which grows downwards). If i is placed at the bottom of the stack, this will likely crash. The compiler is free to put the variables on the stack in any order, so whether it actually crashes will depend heavily on the specific compiler you are using.

You can also see more clearly why the allocation won't crash:

; Just an integer subtraction. Why would it crash?
sub rsp 1073741720