RuntimeWarning: overflow encountered in exp in computing the logistic function

Question

I'm getting this error when I try to compute the logistic function for a data mining method I'm implementing:

RuntimeWarning: overflow encountered in exp

My code:

def logistic_function(x):
#     x = np.float64(x)
    return 1.0 / (1.0 + np.exp(-x))

If I understood correctly from some related questions that the problem is that np.exp() is returning a huge value. I saw suggestions to let numpy ignore the warnings, but the problem is that when I get this error, then the results of my method are horrible. However when I don't get it, then they are as expected. So making numpy ignoring the warning is not a solution for me at all. I don't know what is wrong or how to deal with.

I don't even know if this is a result of a bug because sometimes I get this error and sometimes not! I went through my code many times and everything looks correct!

Answer 1

You should compute the logistic function using either scipy.special.expit, which in recent enough SciPy is more stable than your solution (although earlier versions got it wrong), or by reducing it to tanh:

def logistic_function(x):
    return .5 * (1 + np.tanh(.5 * x))

This version of the function is stable, fast, and fairly accurate.