Floating point mismatch between compilers (Visual Studio 2010 and GCC)

Question

I'm trying to solve a cross-platform issue that's cropping up and I'm not sure quite how to go about it. Here's a demonstration program:

#include <cmath>
#include <cstdio>

int main()
{
    int xm = 0x3f18492a;
    float x = *(float*)&xm;
    x = (sqrt(x) + 1) / 2.0f;
    printf("%f %x\n", x, *(int*)&x);
}

The output on Windows when compiled in VS2010 is:

0.885638 3f62b92a

The output when compiled with GCC 4.8.1 (ideone.com sample) is:

0.885638 3f62b92b

These small mismatches end up ballooning into a serious problem over the course of a program that needs to run identically on multiple platforms. I'm not concerned so much about "accuracy" as that the results match each other. I tried switching the /fp mode in VS to strict from precise, but that doesn't seem to fix it.

What other avenues should I look at to make this calculation have the same result on both platforms?

UPDATE: Interestingly, if I change the code like this, it matches across the platforms:

#include <cmath>
#include <cstdio>

int main()
{
    int xm = 0x3f18492a;
    float x = *(float*)&xm;
    //x = (sqrt(x) + 1) / 2.0f;
    float y = sqrt(x);
    float z = y + 1;
    float w = z / 2.0f;
    printf("%f %x %f %x %f %x %f %x\n", x, *(int*)&x, y, *(int*)&y, z, *(int*)&z, w, *(int*)&w);
}

I'm not sure it's realistic, however, to be walking through the code and changing all floating point operations like this!

Answer 1

Summary: This is generally not supported by compilers, you will have a tough time doing it in a higher-level language, and you will need to use one math library common to all your target platforms.

The C and C++ language standards allow implementations a considerable amount (too much) of flexibility in floating-point operations. Many C and C++ floating-operations are not required to adhere to the IEEE 754-2008 standard in the way that might be intuitive to many programmers.

Even many C and C++ implementations do not provide good support for adhering to the IEEE 754-2008 standard.

Math library implementations are a particular problem. There does not exist any normal library (commercially available or widely-used open source with a known-bounded run-time) that provides correctly rounded results for all standard math functions. (Getting the mathematics right on some of the functions is a very difficult problem.)

sqrt, however, is relatively simple and should return correctly rounded results in an library of reasonable quality. (I am unable to vouch for the Microsoft implementation.) It is more likely the particular problem in the code you show is the compiler’s choice to use varying precisions of floating-point while evaluating expressions.

There may be various switches you can use with various compilers to ask them to conform to certain rules about floating-point behavior. Those may be sufficient for getting elementary operations to perform as expected. If not, assembly language is a way to access well-defined floating-point operations. However, the behavior of library routines will be different between platforms unless you supply a common library. This includes both math library routines (such as pow) and conversions found in routines such as fprintf, fscanf, strtof. You must therefore find one well-designed implementation of each routine you rely on that is supported on all of the platforms you target. (It must be well-designed in the sense that it provides identical behavior on all platforms. Mathematically, it could be somewhat inaccurate, as long as it is within bounds tolerable for your application.)

Answer 2

The Visual Studio compiler tends to generate instructions that use the old x87 FPU(*), but it generates code at the beginning of the executable to set the FPU to the precision of the double format.

GCC can also generate instructions that use the old x87 FPU, but when generating x86-64 code, the default is to use SSE2. On Mac OS X, the default is to use SSE2 even in 32-bit since all Intel Macs have SSE2. When it generates instruction for the 387, GCC does not set the precision of the FPU to the double format, so that computations are made in the 80-bit double-extended format, and then rounded to double when assigned.

As a consequence:

1. If you use only double computations, Visual Studio should generate a program that computes exactly at the precision of the type, because it is always double(**). And if on the GCC side you use -msse2 -mfpmath=sse, you can expect GCC to also generate code that computes at the precision of doubles, this time by using SSE2 instructions. The computations should match.

2. Or if you make both GCC and Visual Studio emit SSE2 instructions, again, the computations should match. I am not familiar with Visual Studio but the switch may be /arch:SSE2.

This does not solve the problem with math libraries, which is indeed an unsolved problem. If your computations involve trigonometric or other functions, you must use the same library as part of your project on both sides. I would recommend CRlibm. Less accurate libraries are fine too as long as it's the same library, and it respects the above constraints (using only double or compiled with SSE2 on both sides).

(*) There may be a way to instruct it to generate SSE2 instructions. If you find it, use it: it will solve your particular problem.

(**) modulo exceptions for infinities and subnormals.