Using Java Double.toString() style formatting with more digits (DecimalFormat does not work with low precision numbers)

Question

The central problem is that I have a series of doubles that I need to log, each with varying number of significant digits. The numbers vary greatly in how many significant digits they have. Some have 0 (e.g. 5257), some have 2 (e.g. 1308.75), some have all the way up to 7 (e.g. 124.1171875). Basically everything between 0 to 7 significant digits after the decimal.

Standard Double.toString() works excellent on everything BUT those with 7 significant digits. That is all the way up to 6 digits, the significant digits are all printed without any insignificant digits. But on those with 7 significant digits, toString() rounds off the last digit. I.e.

5257 -> "5257"
1308.75 -> "1308.75"
124.1171875 -> "124.117188"

Of course I tried using DecimalFormat("#.#######"), and that resolved the problem of missing significant digits, but it printed insignificant digits for many of the low precision doubles. I.e.

1308.75 -> "1308.7499998"

This is also unacceptable as 1) it wastes a significant amount of space (typically log >2 GB of data a day) and 2) it messes up the applications using the logs.

DecimalFormat seems to suck compared to toString() when it comes to identifying significant digits, is there anyway to fix it? I just want to use toString() style handling of significant digits, and extend the maximum number of digits from 6 to 7.

Any ideas? Thanks

Answer 1

If you're concerned about preserving decimal values exactly, you should use BigDecimal to start with - double is fundamentally inappropriate, as a binary floating point type.

As it happens, I can't reproduce your behaviour of 1308.75 in DecimalFormat, which doesn't surprise me because that value is exactly representable as a double. In fact, DecimalFormat appears to be applying some heuristics anyway, as even 1308.76 comes out as 1308.76 - which surprises me as the actual value is 1308.759999999999990905052982270717620849609375. It does mean that if you use 1308.7599999999999 in your code, it will come out as 1308.76, because it's the exact same value as far as double is concerned. If you need to distinguish between those two values, you should definitely be using BigDecimal.

Also note that 1308.75 has 6 significant digits - it has 2 decimal places. It's worth being careful to differentiate between the two concepts.

Answer 2

It seemed to me a bit strange, that's why I went out and tested. I tried this code:

public class MySimpleTest
{
    public static void main(String args[])
    {
        format(5257);
        format(1308.75);
        format(124.1171875);
    }

    private static void format(double d)
    {
        DecimalFormat df = new DecimalFormat("##.#######");
        System.out.println("" + d + " -> " + df.format(d));
    }
}

And it gave me this result:

5257.0 -> 5257
1308.75 -> 1308.75
124.1171875 -> 124.1171875

You probably were testing with "##.######" (only 6 #s after dot), or your number might have had trailing digits. The point is that ##.####### and ##.0000000 formats will round the last decimal point (e.g. 124.11718755 will be rounded to 124.1171876 before formatting). If you want it to be truncated try to truncate it first, doing something like this:

double d = 124.1171875999999;
org.apache.commons.math.util.MathUtils.round(d, 6, java.math.BigDecimal.ROUND_DOWN);
DecimalFormat df = new DecimalFormat("##.#######");
System.out.println("" + d + " -> " + df.format(d));