Why am I not getting a stack overflow?

Question

EDIT Before you got excited see important edits in the end and if you're still curious these are reported as:

• http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=696222
• http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=696263

---

I have been trying a piece of code and surprised to see that I didn't get a stackoverflow. Trying to simplify things I even got this far:

#include <stdio.h>

int main()
{
    int i;

    /* 1,500,000,000 x 4 bytes = 6,000,000,000 bytes = 6GB */
    int size = 1500000000;
    int arr[size];
    for (i = 0; i < size; i++) {
        arr[i] = 1;
    }
    printf("first: %d\n", arr[0]);
    printf("last:  %d\n", arr[size - 1]);

    return 0;
}

which made me doubt that I don't even know the basics of memory management. I was thinking arr[size] should allocate on stack and overflow easily but instead it it uses all my memory and swap and finishes successfully. What am I missing?

Notes

• I'm running on 64 bit ubuntu 12.04

• I have tried with gcc and clang with versions:

  gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
  Ubuntu clang version 3.0-6ubuntu3 (tags/RELEASE_30/final) (based on LLVM 3.0)
  

• I have turned the optimizations off (-O0)

• Running ulimit -a I get:

  core file size          (blocks, -c) 0
  data seg size           (kbytes, -d) unlimited
  scheduling priority             (-e) 0
  file size               (blocks, -f) unlimited
  pending signals                 (-i) 29569
  max locked memory       (kbytes, -l) 64
  max memory size         (kbytes, -m) unlimited
  open files                      (-n) 1024
  pipe size            (512 bytes, -p) 8
  POSIX message queues     (bytes, -q) 819200
  real-time priority              (-r) 0
  stack size              (kbytes, -s) 8192
  cpu time               (seconds, -t) unlimited
  max user processes              (-u) 29569
  virtual memory          (kbytes, -v) unlimited
  file locks                      (-x) unlimited
  

• I have 4GB of ram and about the same amount of swap

---

(gdb) disassemble main
Dump of assembler code for function main:
   0x00000000004004f4 <+0>:     push   %rbp
   0x00000000004004f5 <+1>:     mov    %rsp,%rbp
   0x00000000004004f8 <+4>:     push   %rbx
   0x00000000004004f9 <+5>:     sub    $0x38,%rsp
   0x00000000004004fd <+9>:     mov    %rsp,%rax
   0x0000000000400500 <+12>:    mov    %rax,%rbx
   0x0000000000400503 <+15>:    movl   $0x59682f00,-0x14(%rbp)
   0x000000000040050a <+22>:    mov    -0x14(%rbp),%eax
   0x000000000040050d <+25>:    movslq %eax,%rdx
   0x0000000000400510 <+28>:    sub    $0x1,%rdx
   0x0000000000400514 <+32>:    mov    %rdx,-0x28(%rbp)
   0x0000000000400518 <+36>:    cltq
   0x000000000040051a <+38>:    shl    $0x2,%rax
   0x000000000040051e <+42>:    lea    0xf(%rax),%rdx
   0x0000000000400522 <+46>:    mov    $0x10,%eax
   0x0000000000400527 <+51>:    sub    $0x1,%rax
   0x000000000040052b <+55>:    add    %rdx,%rax
   0x000000000040052e <+58>:    movq   $0x10,-0x38(%rbp)
   0x0000000000400536 <+66>:    mov    $0x0,%edx
   0x000000000040053b <+71>:    divq   -0x38(%rbp)
   0x000000000040053f <+75>:    imul   $0x10,%rax,%rax
   0x0000000000400543 <+79>:    sub    %rax,%rsp
   0x0000000000400546 <+82>:    mov    %rsp,%rax
   0x0000000000400549 <+85>:    add    $0xf,%rax
   0x000000000040054d <+89>:    shr    $0x4,%rax
   0x0000000000400551 <+93>:    shl    $0x4,%rax
   0x0000000000400555 <+97>:    mov    %rax,-0x20(%rbp)
   0x0000000000400559 <+101>:   movl   $0x0,-0x18(%rbp)
   0x0000000000400560 <+108>:   jmp    0x400577 <main+131>
   0x0000000000400562 <+110>:   mov    -0x20(%rbp),%rax
   0x0000000000400566 <+114>:   mov    -0x18(%rbp),%edx
   0x0000000000400569 <+117>:   movslq %edx,%rdx
   0x000000000040056c <+120>:   movl   $0x1,(%rax,%rdx,4)
   0x0000000000400573 <+127>:   addl   $0x1,-0x18(%rbp)
   0x0000000000400577 <+131>:   mov    -0x18(%rbp),%eax
   0x000000000040057a <+134>:   cmp    -0x14(%rbp),%eax
   0x000000000040057d <+137>:   jl     0x400562 <main+110>
   0x000000000040057f <+139>:   mov    -0x20(%rbp),%rax
   0x0000000000400583 <+143>:   mov    (%rax),%edx
   0x0000000000400585 <+145>:   mov    $0x4006bc,%eax
   0x000000000040058a <+150>:   mov    %edx,%esi
   0x000000000040058c <+152>:   mov    %rax,%rdi
   0x000000000040058f <+155>:   mov    $0x0,%eax
---Type <return> to continue, or q <return> to quit---
   0x0000000000400594 <+160>:   callq  0x4003f0 <printf@plt>
   0x0000000000400599 <+165>:   mov    -0x14(%rbp),%eax
   0x000000000040059c <+168>:   lea    -0x1(%rax),%edx
   0x000000000040059f <+171>:   mov    -0x20(%rbp),%rax
   0x00000000004005a3 <+175>:   movslq %edx,%rdx
   0x00000000004005a6 <+178>:   mov    (%rax,%rdx,4),%edx
   0x00000000004005a9 <+181>:   mov    $0x4006c7,%eax
   0x00000000004005ae <+186>:   mov    %edx,%esi
   0x00000000004005b0 <+188>:   mov    %rax,%rdi
   0x00000000004005b3 <+191>:   mov    $0x0,%eax
   0x00000000004005b8 <+196>:   callq  0x4003f0 <printf@plt>
   0x00000000004005bd <+201>:   mov    $0x0,%eax
   0x00000000004005c2 <+206>:   mov    %rbx,%rsp
   0x00000000004005c5 <+209>:   mov    -0x8(%rbp),%rbx
   0x00000000004005c9 <+213>:   leaveq
   0x00000000004005ca <+214>:   retq
End of assembler dump.

---

$ pmap 2840
2840:   ./a.out
0000000000400000      4K r-x--  /home/gokce/play/a.out
0000000000600000      4K r----  /home/gokce/play/a.out
0000000000601000      4K rw---  /home/gokce/play/a.out
00002b572d7be000    136K r-x--  /lib/x86_64-linux-gnu/ld-2.15.so
00002b572d7e0000      8K rw---    [ anon ]
00002b572d9e0000      4K r----  /lib/x86_64-linux-gnu/ld-2.15.so
00002b572d9e1000      8K rw---  /lib/x86_64-linux-gnu/ld-2.15.so
00002b572d9e3000   1748K r-x--  /lib/x86_64-linux-gnu/libc-2.15.so
00002b572db98000   2044K -----  /lib/x86_64-linux-gnu/libc-2.15.so
00002b572dd97000     16K r----  /lib/x86_64-linux-gnu/libc-2.15.so
00002b572dd9b000      8K rw---  /lib/x86_64-linux-gnu/libc-2.15.so
00002b572dd9d000     28K rw---    [ anon ]
00007ffe080a2000 5859388K rw---    [ stack ]
00007fff6dbfc000      4K r-x--    [ anon ]
ffffffffff600000      4K r-x--    [ anon ]
 total          5863408K

IMPORTANT EDIT

I was using a small hand written makefile:

build:
        gcc foo.c -Wall -Wextra -g

run:
        ./a.out

.SILENT:

to run the program using my common editor shotcuts and I realized now that it is somehow relevant. I got the segfault when I run from the console using:

./a.out

but not when I run with:

make run

How is that relevant?

IMPORTANT EDIT2

When I try to run ulimit -s in make run like:

build:
        gcc foo.c -Wall -Wextra -g

run:
        ulimit -s

.SILENT:

it gives:

make: ulimit: Command not found
make: *** [run] Error 127

then I realized it changes when I add an extra # in the end: (isn't it the comment character?)

build:
        gcc foo.c -Wall -Wextra -g

run:
        ulimit -s #

.SILENT:

I get:

unlimited

I also checked my bash aliases and there's no make. which make gives /usr/bin/make and file /usr/bin/make gives:

/usr/bin/make: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically 
linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=0x1d76b881b71091d
37e6653d7c8b8e19a2a414591, stripped

Answer 1

arr is clearly getting allocated on the stack, as you'd expect. From the pmap output, the stack is clearly growing to nearly 6GB:

00007ffe080a2000 5859388K rw---    [ stack ]

The question is therefore not about your program or the compiler. The question is why ulimit -s 8192 is apparently not being enforced.

From your answers to my various question, it is clear that somehow the ulimit -s setting is not being propagated from your shell through make run to your program. This to me seems very odd.

If I were in your shoes, I'd go through the system's limits.conf as well as the shared and the user's bash startup files to see if I can spot anything of relevance.