Why do very large stack allocations fail despite unlimited ulimit?

Question

The following static allocation gives segmentation fault

double U[100][2048][2048];

But the following dynamic allocation goes fine

double ***U = (double ***)malloc(100 * sizeof(double **));

for(i=0;i<100;i++)
{
    U[i] = (double **)malloc(2048 * sizeof(double *));
    for(j=0;j<2048;j++)
    {
        U[i][j] = (double *)malloc(2048*sizeof(double));
    }
}

The ulimit is set to unlimited in linux.

Can anyone give me some hint on whats happening?

Answer 1

When you say the ulimit is set to unlimited, are you using the -s option? As otherwise this doesn't change the stack limit, only the file size limit.

There appear to be stack limits regardless, though. I can allocate:

double *u = malloc(200*2048*2048*(sizeof(double)));  // 6gb contiguous memory

And running the binary I get:

VmData:  6553660 kB

However, if I allocate on the stack, it's:

double u[200][2048][2048];

VmStk:   2359308 kB

Which is clearly not correct (suggesting overflow). With the original allocations, the two give the same results:

Array:  VmStk:   3276820 kB
malloc: VmData:  3276860 kB

However, running the stack version, I cannot generate a segfault no matter what the size of the array -- even if it's more than the total memory actually on the system, if -s unlimited is set.

EDIT:

I did a test with malloc in a loop until it failed:

VmData: 137435723384 kB  // my system doesn't quite have 131068gb RAM

Stack usage never gets above 4gb, however.

Answer 2

Assuming your machine actually has enough free memory to allocate 3.125 GiB of data, the difference most likely lies in the fact that the static allocation needs all of this memory to be contiguous (it's actually a 3-dimensional array), while the dynamic allocation only needs contiguous blocks of about 2048*8 = 16 KiB (it's an array of pointers to arrays of pointers to quite small actual arrays).

It is also possible that your operating system uses swap files for heap memory when it runs out, but not for stack memory.