What are some common strategies different compilers use to deal with overflow in numeric conversions?

Question

I understand that, in C++, when I convert a float/double into an int, whereby the floating-point number is beyond the range that the int can hold, the result is not defined as part of the C++ language. The result depends on the implementation/compiler. What are some strategies common compilers use to deal with this?

Converting 7.2E12 to an int can yield the values 1634811904 or 2147483647. For example, does anyone know what the compiler is doing in each of these cases?

Answer 1

The compiler generates sequences of instructions that produce the correct result for all inputs that do not cause overflow. This is all it has to worry about (because overflow in the conversion from floating-point to integer is undefined behavior). The compiler does not “deal with” overflows so much as completely ignore them. If the underlying assembly instruction(s) on the platform raise an exception, fine. If they wrap around, fine. If they produce nonsensical results, again, fine.

---

As an example, constant expressions may be converted to integers at compile-time with rules that differ from the behavior of the assembly instructions generated on the platform. My blog post gives the example:

int printf(const char *, ...);

volatile double v = 0;

int main()
{
  int i1 = 2147483648.0;
  int i2 = 2147483648.0 + v;
  printf("%d %d\n", i1, i2);
}

which produces a program that prints two different values for i1 and i2. This is because the conversion in the computation of i1 was applied at compile-time, whereas the conversion in the computation of i2 was applied at run-time.

---

As another example, in the particular case of the conversion from double to 32-bit unsigned int on the x86-64 platform, the results can be funny:

There are no instructions in the x86 instruction sets to convert from floating-point to unsigned integer.

On Mac OS X for Intel, compiling a 64-bit program, the conversion from double to 32-bit unsigned int is compiled in a single instruction: the instruction for 64-bit conversions, cvttsd2siq, with destination a 64-bit register of which only the bottom 32-bit will subsequently be used as the 32-bit unsigned integer it represents:

$ cat t.c
#include <stdio.h>
#include <stdlib.h>
int main(int c, char **v)
{
  unsigned int i = 4294967296.0 + strtod(v[1], 0);
  printf("%u\n", i);
}
$ gcc -m64 -S -std=c99 -O t.c && cat t.s
…
addsd LCPI1_0(%rip), %xmm0 ; this is the + from the C program
cvttsd2siq %xmm0, %rsi ; one-instruction conversion
…

This explains how, on that platform, a result modulo 2³² can be obtained for doubles that are small enough (specifically, small enough to fit in a signed 64-bit integer).

In the old IA-32 instruction set, there is no instruction to convert a double to a 64-bit signed integer (and there is no instruction to convert a double to a 32-bit unsigned int either). The conversion to 32-bit unsigned int has to be done by combining a few of the instructions that do exist, including two instructions cvttsd2si to convert from double to 32-bit signed integer:

$ gcc -m32 -S -std=c99 -O t.c && cat t.s
…
addsd LCPI1_0-L1$pb(%esi), %xmm0 ; this is the + from the C program
movsd LCPI1_1-L1$pb(%esi), %xmm1 ; conversion to unsigned int starts here
movapd %xmm0, %xmm2
subsd %xmm1, %xmm2
cvttsd2si %xmm2, %eax
xorl $-2147483648, %eax
ucomisd %xmm1, %xmm0
cvttsd2si %xmm0, %edx
cmovael %eax, %edx
…

Two alternative solutions are computed, respectively in %eax and in %edx. The alternatives are each correct on different definition domains. If the number to convert, in %xmm0, is larger than the constant 2³¹ in %xmm1, then one alternative is chosen, otherwise, the other one is. The high-level algorithm, using only conversions from double to int, would be:

if (d < 231)
then (unsigned int)(int)d
else (231 + (unsigned int)(int)(d - 231))

This translation of the C conversion from double to unsigned int gives the same saturating behavior as the 32-bit conversion instruction that it relies on:

$ gcc -m32 -std=c99 -O t.c && ./a.out 123456
0