Custom loss function in Keras based on the input data

Question

I am trying to create the custom loss function using Keras. I want to compute the loss function based on the input and predicted the output of the neural network.

I tried using the customloss function in Keras. I think y_true is the output that we give for training and y_pred is the predicted output of the neural network. The below loss function is same as "mean_squared_error" loss in Keras.

def customloss(y_true, y_pred):
    return K.mean(K.square(y_pred - y_true), axis=-1)

I would like to use the input to the neural network also to compute the custom loss function in addition to mean_squared_error loss. Is there a way to send an input to the neural network as an argument to the customloss function.

Thank you.

Answer 1

I have come across 2 solutions to the question you asked.

1. You can pass your input tensor as an argument to the custom loss wrapper function.

    def custom_loss(i):

        def loss(y_true, y_pred):
            return K.mean(K.square(y_pred - y_true), axis=-1) + something with i...
        return loss

    def baseline_model():
        # create model
        i = Input(shape=(5,))
        x = Dense(5, kernel_initializer='glorot_uniform', activation='linear')(i)
        o = Dense(1, kernel_initializer='normal', activation='linear')(x)
        model = Model(i, o)
        model.compile(loss=custom_loss(i), optimizer=Adam(lr=0.0005))
        return model

This solution is also mentioned in the accepted answer here

2. You can pad your label with extra data columns from input and write a custom loss. This is helpful if you just want one/few feature column(s) from your input.

    def custom_loss(data, y_pred):

        y_true = data[:, 0]
        i = data[:, 1]
        return K.mean(K.square(y_pred - y_true), axis=-1) + something with i...


    def baseline_model():
        # create model
        i = Input(shape=(5,))
        x = Dense(5, kernel_initializer='glorot_uniform', activation='linear')(i)
        o = Dense(1, kernel_initializer='normal', activation='linear')(x)
        model = Model(i, o)
        model.compile(loss=custom_loss, optimizer=Adam(lr=0.0005))
        return model


    model.fit(X, np.append(Y_true, X[:, 0], axis =1), batch_size = batch_size, epochs=90, shuffle=True, verbose=1)

This solution can be found also here in this thread.

I have only used the 2nd method when I had to use input feature columns in the loss. I have used the first method with scalar arguments; but I believe a tensor input works as well.

Answer 2

You could wrap your custom loss with another function that takes the input tensor as an argument:

def customloss(x):
    def loss(y_true, y_pred):
        # Use x here as you wish
        err = K.mean(K.square(y_pred - y_true), axis=-1)
        return err

    return loss

And then compile your model as follows:

model.compile('sgd', customloss(x))

where x is your input tensor.

NOTE: Not tested.