Final, non-canonical NaN double value changes during runtime

Question

I am writing Java code that interacts with R, where "NA" values are distinguished from NaN values. NA indicates that a value is "statistically missing", that is it could not collected or is otherwise not available.

class DoubleVector {
     public static final double NA = Double.longBitsToDouble(0x7ff0000000001954L);

     public static boolean isNA(double input) {
         return Double.doubleToRawLongBits(input) == Double.doubleToRawLongBits(NA);
     }

     /// ... 
}

The following unit test demonstrates the relationship between NaN and NA and runs fine on my windows laptop but "isNA(NA) #2" fails sometimes on my ubuntu workstation.

@Test
public void test() {

    assertFalse("isNA(NaN) #1", DoubleVector.isNA(DoubleVector.NaN));
    assertTrue("isNaN(NaN)", Double.isNaN(DoubleVector.NaN));
    assertTrue("isNaN(NA)", Double.isNaN(DoubleVector.NA));
    assertTrue("isNA(NA) #2", DoubleVector.isNA(DoubleVector.NA));
    assertFalse("isNA(NaN)", DoubleVector.isNA(DoubleVector.NaN));
}

From debugging, it appears that DoubleVector.NA is changed to the canonical NaN value 7ff8000000000000L, but it's hard to tell because printing it to stdout gives different values than the debugger.

Also, the test only fails if it runs after a number of other previous tests; if I run this test alone, it always passes.

Is this a JVM bug? A side effect of optimization?

Tests always pass on:

java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) Client VM (build 19.1-b02, mixed mode, sharing)

Tests sometimes fail on:

java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)

Answer 1

You are treading in very dangerous water here, one of the few areas where the Java VM behaviour is not exactly specified.

According to the JVM spec, there is only "a NaN value" in the double range. No arithmetic operation on doubles could distinguish between two different NaN values.

The documentation of longBitsToDouble() has this note:

Note that this method may not be able to return a double NaN with exactly same bit pattern as the long argument. IEEE 754 distinguishes between two kinds of NaNs, quiet NaNs and signaling NaNs. The differences between the two kinds of NaN are generally not visible in Java. Arithmetic operations on signaling NaNs turn them into quiet NaNs with a different, but often similar, bit pattern. However, on some processors merely copying a signaling NaN also performs that conversion. In particular, copying a signaling NaN to return it to the calling method may perform this conversion. So longBitsToDouble may not be able to return a double with a signaling NaN bit pattern. Consequently, for some long values, doubleToRawLongBits(longBitsToDouble(start)) may not equal start. Moreover, which particular bit patterns represent signaling NaNs is platform dependent; although all NaN bit patterns, quiet or signaling, must be in the NaN range identified above.

So assuming that handling a double value will always keep the specific NaN value intact is a dangerous thing.

The cleanest solution would be to store your data in long and convert to double after checking for your special value. This will impose a quite noticeable performance impact, however.

You might get away by adding the strictfp flag at the affected places. This doesn't in any way guarantee that it will work, but it will (possibly) change how your JVM handles floating point values and might just be the necessary hint that helps. It will still not be portable, however.