How can I tell whether a binary is compiled with frame pointers or not on Linux?

Question

I have a binary/library in Linux. How can I determine whet it was compiled with frame pointers?

Answer 1

Zan/blackwing:

Compiling some simple things with / without framepointer optimization and using diff -u on the disassembled output gives some clues:

$ diff -u with*
--- with-fp 2011-03-23 09:49:29.366277002 +0000
+++ without-fp  2011-03-23 09:49:35.046277002 +0000
@@ -5,14 +5,12 @@
 Disassembly of section .text:

 00000000 <func>:
-   0:  55                      push   %ebp
+   0:  53                      push   %ebx
    1:  31 c0                   xor    %eax,%eax
-   3:  89 e5                   mov    %esp,%ebp
-   5:  53                      push   %ebx
-   6:  81 ec 00 04 00 00       sub    $0x400,%esp
-   c:  8b 4d 08                mov    0x8(%ebp),%ecx
-   f:  8d 9d fc fb ff ff       lea    -0x404(%ebp),%ebx
-  15:  8d 76 00                lea    0x0(%esi),%esi
+   3:  81 ec 00 04 00 00       sub    $0x400,%esp
+   9:  8b 8c 24 08 04 00 00    mov    0x408(%esp),%ecx
+  10:  89 e3                   mov    %esp,%ebx
+  12:  8d b6 00 00 00 00       lea    0x0(%esi),%esi
   18:  8b 14 81                mov    (%ecx,%eax,4),%edx
   1b:  89 14 83                mov    %edx,(%ebx,%eax,4)
   1e:  83 c0 01                add    $0x1,%eax
@@ -28,5 +26,4 @@
   3e:  75 f0                   jne    30 <func+0x30>
   40:  81 c4 00 04 00 00       add    $0x400,%esp
   46:  5b                      pop    %ebx
-  47:  5d                      pop    %ebp
-  48:  c3                      ret    
+  47:  c3                      ret    

You see multiple kinds of changes:

1. Code with framepointers will always contain the both of the two instructions push %ebp and mov %esp, %ebp.
   Framepointer-less code may (does not in the shown case, as it's not using the %ebp register for anything) have the push %ebp but will not have the mov %esp, %ebp one, as it's not necessary to initialize the framepointer.
2. Code with framepointers accesses arguments on the stack relative to the framepointer, like mov 0x8(%ebp), %ecx in the shown case.
   Framepointer-less code does so relative to the stackpointer, with an additional offset the size of the function's stackframe, like mov 0x408(%esp), %ecx.
   The same might be true for local variables, in the shown code that's lea -0x404(%ebp), %ebx for the framepointer-using code vs. mov %esp, %ebx (could've been lea 0x0(%esp), %ebx) for the code without framepointers.
3. There are likely some changes in register allocation between the two, especially if the code becomes complex enough to use the %ebp register for a local variable (the shown sample isn't showing that)

Compiler optimization levels have quite some impact on how the generated code actually looks like but these specific items mentioned (mov %esp, %ebp and the use of %ebp-relative addressing for arguments / local variables) are only ever found in code that uses framepointers, and missing if you compiled with -fomit-frame-pointer.