SIGSEGV in optimized version of code

Question

My knowledge of the intel instruction set is a bit rusty. Can you tell me why I might be getting a segmentation fault in the optimized version of my function (bonus points if you can tell me why I don't get it in the -O0 build of the code.

It's C code compiled by GCC 4.1.2.

Here is the result of GDB's "disas" command at the crash:

   0x00000000004263e5 <+0>:     sub    $0x8,%rsp
   0x00000000004263e9 <+4>:     movsd  %xmm2,(%rsp)
   0x00000000004263ee <+9>:     divsd  %xmm1,%xmm0
   0x00000000004263f2 <+13>:    callq  0x60f098 <log@plt>
=> 0x00000000004263f7 <+18>:    andpd  0x169529(%rip),%xmm0        
   0x00000000004263ff <+26>:    movsd  (%rsp),%xmm1
   0x0000000000426404 <+31>:    ucomisd %xmm0,%xmm1
   0x0000000000426408 <+35>:    seta   %al
   0x000000000042640b <+38>:    movzbl %al,%eax
   0x000000000042640e <+41>:    add    $0x8,%rsp
   0x0000000000426412 <+45>:    retq   

And here's the original source of the function:

char is_within_range(double a, double b, double range) {
  double ratio = a / b;
  double logRatio = fabs(log(ratio));
  return logRatio < range;
}

For reference here's the non-optimized version of the code:

   0x00000000004263e5 <+0>: push   %rbp
   0x00000000004263e6 <+1>: mov    %rsp,%rbp
   0x00000000004263e9 <+4>: sub    $0x30,%rsp
   0x00000000004263ed <+8>: movsd  %xmm0,-0x18(%rbp)
   0x00000000004263f2 <+13>:    movsd  %xmm1,-0x20(%rbp)
   0x00000000004263f7 <+18>:    movsd  %xmm2,-0x28(%rbp)
   0x00000000004263fc <+23>:    movsd  -0x18(%rbp),%xmm0
   0x0000000000426401 <+28>:    divsd  -0x20(%rbp),%xmm0
   0x0000000000426406 <+33>:    movsd  %xmm0,-0x10(%rbp)
   0x000000000042640b <+38>:    mov    -0x10(%rbp),%rax
   0x000000000042640f <+42>:    mov    %rax,-0x30(%rbp)
   0x0000000000426413 <+46>:    movsd  -0x30(%rbp),%xmm0
   0x0000000000426418 <+51>:    callq  0x610608 <log@plt>
   0x000000000042641d <+56>:    movapd %xmm0,%xmm1
   0x0000000000426421 <+60>:    movsd  0x16b6b7(%rip),%xmm0
   0x0000000000426429 <+68>:    andpd  %xmm1,%xmm0
   0x000000000042642d <+72>:    movsd  %xmm0,-0x8(%rbp)
   0x0000000000426432 <+77>:    movsd  -0x8(%rbp),%xmm1
   0x0000000000426437 <+82>:    movsd  -0x28(%rbp),%xmm0
   0x000000000042643c <+87>:    ucomisd %xmm1,%xmm0
   0x0000000000426440 <+91>:    seta   %al
   0x0000000000426443 <+94>:    movzbl %al,%eax
   0x0000000000426446 <+97>:    leaveq 
   0x0000000000426447 <+98>:    retq   

Answer 1

=> 0x00000000004263f7 <+18>:    andpd  0x169529(%rip),%xmm0        
   0x00000000004263ff <+26>:    movsd  (%rsp),%xmm1

When the andpd instruction takes a memory operand, it's required to be aligned to a 16-byte boundary.

For %rip-relative addressing, the offset is applied to the address of the following instruction. So, here, the memory operand is at 0x4263ff + 0x169529 = 0x58f928, which is not 16-byte aligned. Hence the segfault.

The compiler is directly generating code for fabs(), using an AND with an appropriate bit mask to clear the sign bit; the bit mask constant value should have been placed at an appropriate offset in a sufficiently aligned data section, but hasn't been. This could be a bug in that (old) version of GCC, or could conceivably be a linker-related issue somewhere else.