Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

OverflowError: long int too large to convert to float in python

I tried to calculate poisson distribution in python as below:

p = math.pow(3,idx) depart = math.exp(-3) * p  depart = depart / math.factorial(idx) 

idx ranges from 0

But I got OverflowError: long int too large to convert to float

I tried to convert depart to float but no results.

like image 515
user2312186 Avatar asked Apr 23 '13 16:04

user2312186


People also ask

How do you fix an overflow error in Python?

These errors can be handled by using exception handling. In the below section we will see about this exception handling. In the above programs, we saw the Overflow error that occurred when the current value exceeds the limit value. So to handle this we have to raise overflowError exception.

What happens when a python int gets too large?

OverflowError: Python int too large to convert to C long is a typical error in python which occurs when you initialize too large integer. Currently, the maximum capacity of an integer is limited to sys. maxint . If you initialize the integers greater than it, an error will be raised.


2 Answers

Factorials get large real fast:

>>> math.factorial(170) 7257415615307998967396728211129263114716991681296451376543577798900561843401706157852350749242617459511490991237838520776666022565442753025328900773207510902400430280058295603966612599658257104398558294257568966313439612262571094946806711205568880457193340212661452800000000000000000000000000000000000000000L 

Note the L; the factorial of 170 is still convertable to a float:

>>> float(math.factorial(170)) 7.257415615307999e+306 

but the next factorial is too large:

>>> float(math.factorial(171)) Traceback (most recent call last):   File "<stdin>", line 1, in <module> OverflowError: long int too large to convert to float 

You could use the decimal module; calculations will be slower, but the Decimal() class can handle factorials this size:

>>> from decimal import Decimal >>> Decimal(math.factorial(171)) Decimal('1241018070217667823424840524103103992616605577501693185388951803611996075221691752992751978120487585576464959501670387052809889858690710767331242032218484364310473577889968548278290754541561964852153468318044293239598173696899657235903947616152278558180061176365108428800000000000000000000000000000000000000000') 

You'll have to use Decimal() values throughout:

from decimal import *  with localcontext() as ctx:     ctx.prec = 32  # desired precision     p = ctx.power(3, idx)     depart = ctx.exp(-3) * p      depart /= math.factorial(idx) 
like image 103
Martijn Pieters Avatar answered Sep 18 '22 15:09

Martijn Pieters


When idx gets large either the math.pow and/or the math.factorial will become insanely large and be unable to convert to a floating value (idx=1000 triggers the error on my 64 bit machine). You'll want to not use the math.pow function as it overflows earlier than the built in ** operator because it tries to keep higher precision by float converting earlier. Additionally, you can wrap each function call in a Decimal object for higher precision.

Another approach when dealing with very large numbers is to work in the log scale. Take the log of every value (or calculate the log version of each value) and perform all required operations before taking the exponentiation of the results. This allows for your values to temporary leave the floating domain space while still accurately computing a final answer that lies within floating domain.

3 ** idx  =>  math.log(3) * idx math.exp(-3) * p  =>  -3 + math.log(p) math.factorial(idx)  =>  sum(math.log(ii) for ii in range(1, idx + 1)) ... math.exp(result) 

This stays in the log domain until the very end so your numbers can get very, very large before you'll hit overflow problems.

like image 22
Pyrce Avatar answered Sep 16 '22 15:09

Pyrce