Is gcc reordering local variables at compilation time?

Question

I'm currently reading (for the second time) "Hacking : The Art of Exploitation" and have stumbled on something.

The book suggests two different ways to exploit these two similar programs : auth_overflow and auth_overflow2

In the first one, there is a password checking function layed out like this

int check_authentication(char *password) {
    int auth_flag = 0;
    char password_buffer[16];

    strcpy(password_buffer, password);
    ...
}

Inputing more than 16 ASCII characters will change the value of auth_flag to something greater than 0, thus bypassing the check, as shown on this gdb output:

gdb$ x/12x $esp
0xbffff400: 0xffffffff  0x0000002f  0xb7e0fd24  0x41414141
0xbffff410: 0x41414141  0x41414141  0x41414141  0x00000001
0xbffff420: 0x00000002  0xbffff4f4  0xbffff448  0x08048556

password_buffer @ 0xbffff40c
auth_flag @ 0xbffff41c

The second program inverts the two variables :

int check_authentication(char *password) {
    char password_buffer[16];
    int auth_flag = 0;

    strcpy(password_buffer, password);
    ...
}

The author then suggests than it's not possible to overflow into auth_flag, which I really believed. I then proceeded to overflow the buffer, and to my surprise, it still worked. The auth_flag variable was still sitting after the buffer, as you can see on this gdb output:

gdb$ x/12x $esp
0xbffff400: 0xffffffff  0x0000002f  0xb7e0fd24  0x41414141
0xbffff410: 0x41414141  0x41414141  0x41414141  0x00000001
0xbffff420: 0x00000002  0xbffff4f4  0xbffff448  0x08048556

password_buffer @ 0xbffff40c
auth_flag @ 0xbffff41c

I'm wondering if gcc is not reordering local variables for alignement/optimization purposes.

I tried to compile using -O0 flag, but the result is the same.

Does one of you knows why is this happening ?

Thanks in advance.

Answer 1

The compiler authors are completely free to implement any allocation scheme for local variables with automatic storage. auth_flag could be set before or after password_buffer on the stack, it could be in a register, it could be elided completely if proper analysis of the code allows it. There might not even be a stack... The only guarantee the Standard gives you is this:

strcpy(password_buffer, password); invokes undefined behavior if the source string including its null terminator is longer than the destination array password_buffer. Whether this undefined behavior fits your needs is completely outside of the language specification.

As a matter of fact, some implementors purposely complicate the task of would be hackers by randomizing the behavior in cases such as the posted code.