Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does Long.MaxValue fit in a Double but < Long.MaxValue not

Tags:

scala

I know there is a simple answer for this but it has me puzzled.

Why is it that Long.MaxValue can be stored in a Double and returned as longValue, but a lesser Long loses precision.

I would have expected a loss of precision after 53 bits ? What is going on here

scala> val x:Double = Long.MaxValue  // x: Double = 9.223372036854776E18    (9223372036854775807)
scala> x == Long.MaxValue            // Boolean = true
scala> x.longValue == Long.MaxValue  // Boolean = true
scala> val y = 1234567890123456789L  // y: Long = 1234567890123456789  // less than Long.MaxValue
scala> y < x                         // Boolean = true
scala> val z:Double = y              // z: Double = 1.23456789012345677E18
scala> z == y                        // Boolean = true
scala> z.longValue == y              // Boolean = false  // why ? MaxValue fits in the Double
scala> z.longValue                   // 1234567890123456768 <-- loss of 21
like image 719
Ramon Avatar asked Mar 16 '23 22:03

Ramon


2 Answers

It's not a matter of "fitting". Doubles are just inherently inexact. Some values will be represented exactly, some won't. It looks like you found one that can be (your x) and one that can't (y).

It's not particular to the values you've chosen. Here's similar behavior with smaller values that clearly "fit" into Doubles:

val a = 1.1        // a: Double = 1.1
a == 1.1           // true

val b = 2.2        // b: Double = 2.2
b == 2.2           // true

val c = 1.1 + 2.2  // c: Double = 3.3000000000000003
c == 3.3           // false
like image 125
dhg Avatar answered Mar 20 '23 04:03

dhg


It's actually not fitting, and it is losing precision.

But the loss is because it's being rounded up:

scala> BigDecimal.exact(Long.MaxValue.toDouble)
res0: scala.math.BigDecimal = 9223372036854775808

scala> Long.MaxValue
res1: Long = 9223372036854775807

What happens when you try to fit a too-big Double into a Long?

scala> 1e100.toLong
res2: Long = 9223372036854775807

It's truncated.

So Long.MaxValue is rounded up to 2^63 which can be represented exactly with 53 bits and a binary exponent, and since it's bigger than Long.MaxValue it turns back into Long.MaxValue when you convert to Long.

like image 31
Rex Kerr Avatar answered Mar 20 '23 04:03

Rex Kerr