Run test vs. Debug test = different results

Question

We have some unit tests that check the result of the solution of linear system of equation, comparing floating point numbers with a delta.

Trying to adjust the delta, I noticed that the same number changes slightly between Visual Studio Run test and Debug test modes.

Why does this happen? When I debug a test the #if DEBUG sections are disabled, therefore the executed code should be the same.

Thanks.

Answer 1

For a simple example of code that produces different results between a typical DEBUG and RELEASE build (unoptimized vs. optimized), try this in LINQPad:

void Main()
{
    float a = 10.0f / 3;
    float b = 10;
    b /= 3;

    (a == b).Dump();
    (a - b).Dump();
}

If you execute this with optimizations on (make sure the little button all the way down to the right in the LINQPad window is turned to "/o+"), you'll get this result:

False
-7,947286E-08

If you disable it, turn off optimizations, you get this:

True
0

Note that the produced IL code is the same:

Note that the addresses differ, this might indicate that there are things here other than just pure IL, though I have no idea what that might be.

Answer 2

There are all sorts of things that can impact floating point computation, the most significant of which is whether it actually writes the value to a local/field or not. It is possible that for the optimized build, the JIT is able to keep the value in a register - the FPU registers are 80 bits wide, to minimize cumulative errors. If it needs to actually write the value down to a 32-bit (float) or 64-bit (double) local or field, it will by necessity lose some of this. So yes, if it can do all the work in registers - it can give a different (usually more "correct") result than if it writes the intermediate values to locals etc.

There are other available registers too, but I doubt these are in use here: XMM/SSE registers are 128 bit; SIMD can (depending on the machine) be up to 512 bit.