How do I implement this metric in Keras? My code below gives the wrong result! Note that I'm undoing a previous log(x + 1) transformation via exp(x) - 1, also negative predictions are clipped to 0:
def rmsle_cust(y_true, y_pred):
first_log = K.clip(K.exp(y_pred) - 1.0, 0, None)
second_log = K.clip(K.exp(y_true) - 1.0, 0, None)
return K.sqrt(K.mean(K.square(K.log(first_log + 1.) - K.log(second_log + 1.)), axis=-1)
For comparison, here's the standard numpy implementation:
def rmsle_cust_py(y, y_pred, **kwargs):
# undo 1 + log
y = np.exp(y) - 1
y_pred = np.exp(y_pred) - 1
y_pred[y_pred < 0] = 0.0
to_sum = [(math.log(y_pred[i] + 1) - math.log(y[i] + 1)) ** 2.0 for i,pred in enumerate(y_pred)]
return (sum(to_sum) * (1.0/len(y))) ** 0.5
What I'm doing wrong? Thanks!
EDIT: Setting axis=0
seems to give a value very close to the correct one, but I'm not sure since all the code I've seem uses axis=-1
.
I ran into the same problem and searched for it, here is what I found
https://www.kaggle.com/jpopham91/rmlse-vectorized
After modified a bit, this seems to work for me,rmsle_K
method implemented with Keras
and TensorFlow
.
import numpy as np
import math
from keras import backend as K
import tensorflow as tf
def rmsle(y, y0):
assert len(y) == len(y0)
return np.sqrt(np.mean(np.power(np.log1p(y)-np.log1p(y0), 2)))
def rmsle_loop(y, y0):
assert len(y) == len(y0)
terms_to_sum = [(math.log(y0[i] + 1) - math.log(y[i] + 1)) ** 2.0 for i,pred in enumerate(y0)]
return (sum(terms_to_sum) * (1.0/len(y))) ** 0.5
def rmsle_K(y, y0):
return K.sqrt(K.mean(K.square(tf.log1p(y) - tf.log1p(y0))))
r = rmsle(y=[5, 20, 12], y0=[8, 16, 12])
r1 = rmsle_loop(y=[5, 20, 12], y0=[8, 16, 12])
r2 = rmsle_K(y=[5., 20., 12.], y0=[8., 16., 12.])
print(r)
print(r1)
sess = tf.Session()
print(sess.run(r2))
Result:
Using TensorFlow backend
0.263978210565
0.263978210565
0.263978
By the use of a list (to_sum
) in the numpy implementation, I suspect your numpy array has shape (length,)
.
And on Keras, since you've got different results with axis=0
and axis=1
, you probably got some shape like (length,1)
.
Also, when creating the to_sum
list, you're using y[i]
and y_pred[i]
, which means you're taking elements from the axis=0
in numpy implementation.
The numpy implementation also sums everything for calculating the mean in sum(to_sum)
. So, you really don't need to use any axis
in the K.mean
.
If you make sure your model's output shape is either (length,)
or (length,1)
, you can use just K.mean(value)
without passing the axis parameter.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With