Significant Figures in JavaScript

Question

I have a few scientists for clients and they have some problems with how toPrecision is rounding in JavaScript. They want everything rounded to a MAX of 3 sig figs which works most of the time but let me give a few examples of what they want:

Lab value to rounded value

123.5 to 124

1234 to 1230

12.0 to 12.0

0.003 to 0.003

So in other words round things with more than 3 sig figs down to 3. If something has 1 or 2 sig figs DONT append a zero (as that implies the lab was more accurate then they really were) but also in the case of 12.0 DONT remove the zero (as that implies the lab is less accurate then they really were)

Answer 1

Using toPrecision works for all examples given except for the 12.0 to 12.0 example:

var nums = [123.5, 1234, 12.0, 0.003]
var out = nums.map(num => parseFloat(num.toPrecision(3)))
// => [124, 1230, 12, 0.003]

It rounds numbers with more than 3 sig figs to 3, but if you use a number with a .0 or .00 on the end it fails. The reason for this is that the JavaScript engine equates 1.00 to 1, and 12.0 to 12, so the problem is actually not toPrecision, but rather JavaScript itself.

To work around this, what you can do is input numbers as strings, and use toPrecision if there isn't a decimal zero, otherwise operate on the string itself:

var nums = ['123.5', '1234', '12.0', '0.003', '1.000', '1236.00'];
var out = nums.map(str => {
    if (/\.0+$/.test(str)) { // test if it ends with .0 or .00, etc.
        // use alternative string method:
        var zeros = str.match(/[0]+$/)[0].length; // count the no. of trailing zeros
        var sigfigs = parseFloat(str).toString().length; // no. of other sig figs
        var zerosNeeded = 3 - sigfigs;
        if (zerosNeeded < 0) {
            return parseFloat(parseFloat(str).toPrecision(3)).toFixed();
        } else {
            return str.substring(0, sigfigs + 1 + zerosNeeded); // +1 for the decimal point
        }
    } else {console.log(str)
        return parseFloat(parseFloat(str).toPrecision(3)).toString()
    }

});
// => ["124", "1230", "12.0", "0.003", "1.00", "1240"]

This works, however as the result must be in a string format if you need to work with floating point numbers and similar, I'd recommend using a different language such as Python. Anyway, I hope this helps!