Keras crossentropy

Question

I'm working with Keras and I'm trying to rewrite categorical_crossentropy by using the Keras abstract backend, but I'm stuck.

This is my custom function, I want just the weighted sum of crossentropy:

def custom_entropy( y_true, y_pred):
    y_pred /= K.sum(y_pred, axis=-1, keepdims=True)

    # clip to prevent NaN's and Inf's
    y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())

   loss = y_true * K.log(y_pred) 
   loss = -K.sum(loss, -1)

   return loss

In my program I generate a label_pred with to model.predict().

Finally I do:

    label_pred = model.predict(mfsc_train[:,:,5])
    cc = custom_entropy(label, label_pred)
    ce = K.categorical_crossentropy(label, label_pred)

I get the following error:

Traceback (most recent call last):
  File "SAMME_train_all.py", line 47, in <module>
    ce = K.categorical_crossentropy(label, label_pred)
  File "C:\Users\gionata\AppData\Local\Programs\Python\Python36\lib
s\keras\backend\tensorflow_backend.py", line 2754, in categorical_c
    axis=len(output.get_shape()) - 1,
AttributeError: 'numpy.ndarray' object has no attribute 'get_shape'

Answer 1

Keras backend functions such K.categorical_crossentropy expect tensors.

It's not obvious from your question what type label is. However, we know that model.predict always returns NumPy ndarrays, so we know label_pred is not a tensor. It is easy to convert, e.g. (assuming label is already a tensor),

custom_entropy(label, K.constant(label_pred))

Since the output of this function is a tensor, to actually evaluate it, you'd call

K.eval(custom_entropy(label, K.constant(label_pred)))

Alternatively, you can just use model as an op, and calling it on a tensor results in another tensor, i.e.

label_pred = model(K.constant(mfsc_train[:,:,5]))
cc = custom_entropy(label, label_pred)
ce = K.categorical_crossentropy(label, label_pred)

Now label_pred, cc and ce will all be tensors.

Answer 2

As given in the documentation, arguments are tensors:

y_true: True labels. TensorFlow/Theano tensor.
y_pred: Predictions. TensorFlow/Theano tensor of the same shape as y_true.

Converting numpy arrays to tensors should solve it.