How to handle single output multiple losses in keras?

Question

My model has a single output. But I want to separate the loss into 3 different components so that I can track the progress of each. Is there a way to do this with keras?

Maybe I could define the same loss components as metrics but is there a more elegant way?

Answer 1

You can define the losses as Keras layers and then you can add all of your losses and metrics (if you want) manually.

You can see a complete tutorial about this topic here

TL;DR:

• Define layers where you calculate the loss
• Write your own compile() function where you add the optimizer, and the losses and the metrics
• At model.compile(optimizer="adam", loss=...) add None as loss

This is how manually adding the losses looks like in code:

loss_layer_names = {"my_loss", ...}

# Adding losses
for name in loss_layer_names:
    layer = model.get_layer(name)
    loss = (tf.reduce_mean(layer.output, keepdims=True))
    model.add_loss(loss)

# Adding metrics
for name in loss_layer_names:
    layer = model.get_layer(name)
    loss = (tf.reduce_mean(layer.output, keepdims=True))
    model.metrics_names.append(name)
    model.metrics_tensors.append(loss)

model.compile(optimizer="adam", loss=[None] * len(model.outputs))

Where model is a Keras model