Is it possible to predict a stack overflow in C on Linux?

Question

There are certain conditions that can cause stack overflows on an x86 Linux system:

• struct my_big_object[HUGE_NUMBER] on the stack. Walking through it eventually causes SIGSEGV.
• The alloca() routine (like malloc(), but uses the stack, automatically frees itself, and also blows up with SIGSEGV if it's too big). Update: alloca() isn't formally deprecated as I originally stated; it is merely discouraged.

Is there a way to programmatically detect if the local stack is big enough for a given object? I know the stack size is adjustable via ulimit, so I have hope there is a way (however non-portable it may be). Ideally, I would like to be able to do something like this:

int min_stack_space_available = /* ??? */;
if (object_size < min_stack_space_available)
{
    char *foo = alloca(object_size);
    do_stuff(foo);
}
else
{
    char *foo = malloc(object_size);
    do_stuff(foo);
    free(foo);
}

Answer 1

You can determine the stack space the process has available by finding the size of a process' stack space and then subtracting the amount used.

ulimit -s

shows the stack size on a linux system. For a programmatic approach, check out getrlimit(). Then, to determine the current stack depth, subtract a pointer to the top of the stack from one to the bottom. For example (code untested):

unsigned char *bottom_of_stack_ptr;

void call_function(int argc, char *argv) {
    unsigned char top_of_stack;
    unsigned int depth = (&top_of_stack > bottom_of_stack_ptr) ? 
        &top_of_stack-bottom_of_stack_ptr : 
        bottom_of_stack_ptr-&top_of_stack;

    if( depth+100 < PROGRAMMATICALLY_DETERMINED_STACK_SIZE ) {
        ...
    }
}

int main(int argc, char *argv) {
    unsigned char bottom_of_stack;
    bottom_of_stack_ptr = &bottom_of_stack;
    my_function();
    return 0;
}