Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

RuntimeWarning: overflow encountered in exp in computing the logistic function

I'm getting this error when I try to compute the logistic function for a data mining method I'm implementing:

RuntimeWarning: overflow encountered in exp

My code:

def logistic_function(x):
#     x = np.float64(x)
    return 1.0 / (1.0 + np.exp(-x))

If I understood correctly from some related questions that the problem is that np.exp() is returning a huge value. I saw suggestions to let numpy ignore the warnings, but the problem is that when I get this error, then the results of my method are horrible. However when I don't get it, then they are as expected. So making numpy ignoring the warning is not a solution for me at all. I don't know what is wrong or how to deal with.

I don't even know if this is a result of a bug because sometimes I get this error and sometimes not! I went through my code many times and everything looks correct!

like image 770
Jack Twain Avatar asked Oct 06 '14 14:10

Jack Twain


People also ask

What does overflow encountered in EXP mean?

This warning occurs when you use the NumPy exp function, but use a value that is too large for it to handle. It's important to note that this is simply a warning and that NumPy will still carry out the calculation you requested, but it provides the warning by default.

How do you avoid overflow in NP EXP?

Fix for Overflow in numpy. We have to store values in a data type capable of holding such large values to fix this issue. For example, np. float128 can hold way bigger numbers than float64 and float32 . All we have to do is just typecast each value of an array to a bigger data type and store it in a numpy array.


1 Answers

You should compute the logistic function using either scipy.special.expit, which in recent enough SciPy is more stable than your solution (although earlier versions got it wrong), or by reducing it to tanh:

def logistic_function(x):
    return .5 * (1 + np.tanh(.5 * x))

This version of the function is stable, fast, and fairly accurate.

like image 175
Fred Foo Avatar answered Oct 12 '22 13:10

Fred Foo