Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Overflow Error in Python's numpy.exp function

I want to use numpy.exp like this:

cc = np.array([     [0.120,0.34,-1234.1] ])  print 1/(1+np.exp(-cc)) 

But this gives me error:

/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:5: RuntimeWarning: overflow encountered in exp 

I can't understand why? How can I fix this? It seems the problem is with third number (-1234.1)

like image 344
Fcoder Avatar asked Nov 21 '16 17:11

Fcoder


People also ask

What does overflow encountered in exp mean?

This warning occurs when you use the NumPy exp function, but use a value that is too large for it to handle. It's important to note that this is simply a warning and that NumPy will still carry out the calculation you requested, but it provides the warning by default.

What does exp do in NumPy?

The exp function in NumPy is used to compute the exponent of all values present in the given array. e refers to Euler's constant. It has an approximate value of 2.718.

How do you stop an exponential overflow?

The idea is that you should avoid to call exp(something) with something being too big. So avoid using exp(x) when x >> 0 and avoid using exp(-x) when x << 0 . In order to achieve that you could start by writing one expression that works with x > 0 and another one that works for x < 0.


1 Answers

As fuglede says, the issue here is that np.float64 can't handle a number as large as exp(1234.1). Try using np.float128 instead:

>>> cc = np.array([[0.120,0.34,-1234.1]], dtype=np.float128) >>> cc array([[ 0.12,  0.34, -1234.1]], dtype=float128) >>> 1 / (1 + np.exp(-cc)) array([[ 0.52996405,  0.58419052,  1.0893812e-536]], dtype=float128) 

Note however, that there are certain quirks with using extended precision. It may not work on Windows; you don't actually get the full 128 bits of precision; and you might lose the precision whenever the number passes through pure python. You can read more about the details here.

For most practical purposes, you can probably approximate 1 / (1 + <a large number>) to zero. That is to say, just ignore the warning and move on. Numpy takes care of the approximation for you (when using np.float64):

>>> 1 / (1 + np.exp(-cc)) /usr/local/bin/ipython3:1: RuntimeWarning: overflow encountered in exp   #!/usr/local/bin/python3.4 array([[ 0.52996405,  0.58419052,  0.        ]]) 

If you want to suppress the warning, you could use scipy.special.expit, as suggested by WarrenWeckesser in a comment to the question:

>>> from scipy.special import expit >>> expit(cc) array([[ 0.52996405,  0.58419052,  0.        ]]) 
like image 64
Praveen Avatar answered Sep 20 '22 01:09

Praveen