Clang producing executable with illegal instruction

Question

I boiled the problem I'm seeing down to a small example. Here is the LLVM assembler code I'm using (in foo.ll):

target datalayout = "e-p:64:64:64-S128-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f16:16:16-f32:32:32-f64:64:64-f128:128:128-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
target triple = "x86_64-pc-linux-gnu"

define fastcc i32 @foo(i32) {
entry:
    %x = add i32 %0, 1
    ret i32 %x
}

define i32 @main(i32, i8**) {
entry:
    %2 = call i32 @foo(i32 %0)
    ret i32 %2
}

I then compile with:

clang -O1 -o foo foo.ll

... and when I run it I get:

Illegal instruction (core dumped)

... so I fire up my debugger, and see this:

Program received signal SIGILL, Illegal instruction.
0x00000000004004d0 in main ()
(gdb) bt
#0  0x00000000004004d0 in main ()
(gdb) disas
Dump of assembler code for function main:
=> 0x00000000004004d0 <+0>: ud2    
End of assembler dump.
(gdb) 

Note that if I change either of the following the program executes fine:

• Remove -O1 from the clang flags
• Remove fastcc from the declaration of @foo in foo.ll

For reference, "clang -v" is:

clang version 3.3 (tags/RELEASE_33/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix

Also, if it helps here is the result of "objdump -d foo".

Answer 1

Your callee is marked "fastcall" but the call is not. The calling conventions needs to match or else it is undefined behaviour which in turn gets optimized down to "ud2", or perhaps nothing at all. This is a FAQ: http://llvm.org/docs/FAQ.html#why-does-instcombine-simplifycfg-turn-a-call-to-a-function-with-a-mismatched-calling-convention-into-unreachable-why-not-make-the-verifier-reject-it

Answer 2

There is a bug where when optimizing function calls, clang will produce an undefined instruction ud2 (which will raise the invalid opcode error) indicate it couldn't handle something.

Essentially, to speed up things, it places the return value in a register. If the return value won't fit in a register (and thus would be returned on the stack), then instead of a ret, it emits a ud2.

This is a known bug (in 3.2 at least).