How to test a custom loss function in keras?

Question

I am training a CovNet with two outputs. My training samples look like this:

[0, value_a1], [0, value_a2], ...

and

[value_b1, 0], [value_b2, 0], ....

I want to generate my own loss function and mask pairs that contain the mask_value = 0. I have this function, though I am not sure whether it really does what I want. So, I want to write some tests.

from tensorflow.python.keras import backend as K
from tensorflow.python.keras import losses

def masked_loss_function(y_true, y_pred, mask_value=0):
    '''
    This model has two target values which are independent of each other.
    We mask the output so that only the value that is used for training 
    contributes to the loss.
        mask_value : is the value that is not used for training
    '''
    mask = K.cast(K.not_equal(y_true, mask_value), K.floatx())
    return losses.mean_squared_error(y_true * mask, y_pred * mask)

Though, I don't know how I can test this function with keras? Usually, this would be passed to model.compile(). Something like along these lines:

x = [1, 0]
y = [1, 1]
assert masked_loss_function(x, y, 0) == 0

Answer 1

I think one way of achieving that is using a Keras backend function. Here we define a function that takes as input two tensors and returns as output a tensor:

from keras import Model
from keras import layers

x = layers.Input(shape=(None,))
y = layers.Input(shape=(None,))
loss_func = K.function([x, y], [masked_loss_function(x, y, 0)])

And now we can use loss_func to run the computation graph we have defined:

assert loss_func([[[1,0]], [[1,1]]]) == [[0]]

Note that keras backend function, i.e. function, expects that the input and output arguments be an array of tensors. Additionally, x and y takes a batch of tensors, i.e. an array of tensors, with undefined shape.

Answer 2

This is another workaround,

x = [1, 0]
y = [1, 1]
F = masked_loss_function(K.variable(x), K.variable(y), K.variable(0))
assert K.eval(F) == 0