gcc memory allocation issue - buffer overflow attack

Question

Does gcc do memory allocation intelligently to prevent buffer overflow attack?

int function(char *str) {
    int a = 0;                 // See the
    char b[16] = "abcd";       // changes here

    if(!strcmp(b, str))
        a = 1;

    return a;
}

and

int function(char *str) {
    char b[16] = "abcd";       // See the
    int a = 0;                 // changes here

    if(!strcmp(b, str))
        a = 1;

    return a;
}

When I debug it with gdb, it always allocate memory first to integer variables and then character array; no matter what is the order of variable declaration. i.e. In above both cases, compiler allocates memory first to a and then to b.

(higher address)
  Memory
|        |
|        |
+--------+
|        |
|        |
|        |
|        |
+--------+ <----- b (16 bytes)
|        |
+--------+ <----- a (4 bytes)
|        |
(lower address)

So, even if we supply more than 16 character in str, it can not affect value of a. Can anybody help me out here?

Thank you.

Answer 1

Yes, if run with the -fstack-protector flag.

When run with the flag, GCC adds stack canaries, sorts array variables to the highest part of the stack frame to make it more difficult to overflow them and corrupt other variables, and makes copies of the function arguments to be stored with the other locals.

See the Wikipedia page on Buffer overflow protection and the ProPolice homepage for more information

Answer 2

Even if GCC has such a feature to protect against buffer overflows, there are many other considerations here that may cause a fixed variable declaration order. Where the declaration is made is not really important, the compiler will take allocation decisions based on when and how the variable is used in runtime.

Most importantly, the compiler will hopefully allocate variables in the stack frame with the best possible alignment in mind. This could be made in entirely different ways depending on CPU and optimizing setting. Optimize for speed may give a completely different allocation, compared to optimize for memory consumption. And most likely, it will put some variables in CPU registers, removing the whole RAM allocation need.

So to answer your question: GCC allocates variables in various ways, depending on compiler port. How it does so, is not something the programmer needs to overly concern themselves about. There may be options to rearrange the stack to protect against buffer overflow attacks, but that only makes sense in some types of applications. There might not even be any input to a particular system, for all we know. So it doesn't make sense for a compiler to have this security feature enabled by default.