64 bit floating point porting issues

Question

I'm porting my application from 32 bit to 64 bit. Currently, the code compiles under both architectures, but the results are different. For various reasons, I'm using floats instead of doubles. I assume that there is some implicit upconverting from float to double happening on one machine and not the other. Is there a way to control for this, or specific gotchas I should be looking for?

edited to add:

32 bit platform

 gcc (GCC) 4.1.2 20070925 (Red Hat 4.1.2-33)
 Dual-Core AMD Opteron(tm) Processor 2218 HE

64 bit platform

 gcc (Ubuntu 4.3.3-5ubuntu4) 4.3.3
 Intel(R) Xeon(R) CPU

Applying the -mfpmath=387 helps somewhat, after 1 iteration of the algorithm the values are the same, but beyond that they fall out of sync again.

I should also add that my concern isn't that the results aren't identical, it's that porting to a 64 bit platform has uncovered a 32 bit dependency of which I was not aware.

Answer 1

Your compiler is probably using SSE opcodes to do most of its floating point arithmetic on the 64 bit platform assuming x86-64, whereas for compatibility reasons it probably used the FPU before for a lot of its operations.

SSE opcodes offer a lot more registers and consistency (values always remain 32 bits or 64 bits in size), while the FPU uses 80 bit intermediate values when possible. So you were most likely benefitting from this improved intermediate precision before. (Note the extra precision can cause inconsistent results like x == y but cos(x) != cos (y) depending on how far apart the computations occur!)

You may try to use -mfpmath=387 for your 64 bit version since you are compiling with gcc and see if your results match your 32 bit results to help narrow this down.

Answer 2

There is no inherent need for floats and doubles to behave differently between 32-bit and 64-bit code but frequently they do. The answer to your question is going to be platform and compiler specific so you need to say what platform you are porting from and what platform you are porting to.

On intel x86 platforms 32-bit code often uses the x87 co-processor instruction set and floating-point register stack for maximum compatibility whereas on amb64/x86_64 platforms, the SSE* instructions and xmm* registers and are often used instead. These have different precision characteristics.

Post edit:

Given your platform, you might want to consider trying the -mfpmath=387 (the default for i386 gcc) on your x86_64 build to see if this explains the differing results. You may also want to look at the settings for all the -fmath-* compiler switches to ensure that they match what you want in both builds.

Answer 3

Like others have said, you haven't provided enough information to tell exactly what's going on. But in a general sense, it seems you've been counting on some kind of floating point behavior that you shouldn't be counting on.

99 times out of 100 the problem is that you're comparing two floats for equality somewhere.

If the problem is simply that you're getting slightly different answers, you need to realize that neither one is "correct" -- some sort of rounding is going to be taking place no matter what architecture you're on. It's a matter of understanding the significant digits in your calculations, and being aware that any values you're coming up with are approximations to a certain degree.

Answer 4

The x87 FPU's 80-bit internal registers cause its floating point results to differ slightly from other FPUs that use 64-bit internally (like on x86_64). You will get different results between these processors unless you don't mind taking large performance hits by flushing things out to memory or doing other "strictfp" tricks.

See also: Floating point rounding when truncating

And: http://docs.sun.com/source/806-3568/ncg_goldberg.html

Answer 5

On x64, the SSE2 instruction set is used, while in 32-bit apps, the x87 FPU is often the default.

The latter internally stores all floating-point values in a 80-bit format. The latter uses plain 32-bit IEEE floats.

Apart from that, an important point to make is that you shouldn't rely on your floating-point math being identical across architectures.

Even if you use 32-bit builds on both machines, there's still no guarantee that Intel and AMD will yield identical results. Of course, when one of them runs a 64-bit build, you only add more uncertainty.

Relying on the precise results of a floating-point operation would almost always be a bug.

Enabling SSE2 on the 32-bit version as well would be a good start, but again, don't make assumptions about floating-point code. There is always a loss of precision, and it's a bad idea to assume that this loss is predictable, or that it can be reproduced between CPU's or different builds.