Custom RMSPE loss function in keras

Question

I am trying to define my own loss function in keras which is Root Mean Squared Percentage Error. RMSPE is defined as :

I have defined my loss function as:

from keras import backend as K
def rmspe(y_true, y_pred):
    sum = K.sqrt(K.mean(K.square( (y_true - y_pred) /
          K.clip(K.abs(y_true),K.epsilon(),None) ), axis=-1) )
    return sum*100.

But after a few iterations it is giving me loss value as nan. Can someone point out what am i doing wrong. Thanks

Answer 1

It's good that you're clipping the denominator. But epsilon in the tensorflow backend is 1e-7 when I check on my machine. So you can still blow up your gradient by ten million when you divide by the clipped value. What you want to do is clip your gradient which you can do with either the clipvalue or clipnorm arguments to your optimizer:

optimizer = SGD(clipvalue=10.0)

or

optimizer = SGD(clipnorm=2.0)

You have to play with the value a bit depending on how many output variables you have and how much noise is in your data.