Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

TensorFlow 2 custom loss: "No gradients provided for any variable" error

I have an image segmentation problem I have to solve in TensorFlow 2.

In particular I have a training set composed by aerial images paired with their respective masks. In a mask the terrain is colored in black and the buildings are colored in white. The purpose is to predict the mask for the images in the test set.

I use a UNet with a final Conv2DTranspose with 1 filter and a sigmoid activation function. The prediction is made in the following way on the output of the final sigmoid layer: if y_pred>0.5, then it's a building, otherwise it's the background.

I want to implement a dice loss, so I wrote the following function

def dice_loss(y_true, y_pred):
    print("[dice_loss] y_pred=",y_pred,"y_true=",y_true)
    y_pred = tf.cast(y_pred > 0.5, tf.float32)
    y_true = tf.cast(y_true, tf.float32)
    numerator = 2 * tf.reduce_sum(y_true * y_pred)
    denominator = tf.reduce_sum(y_true + y_pred)

    return 1 - numerator / denominator

which I pass to TensorFlow in the following way:

loss = dice_loss
optimizer = tf.keras.optimizers.Adam(learning_rate=config.learning_rate)
metrics = [my_IoU, 'acc']
model.compile(optimizer=optimizer, loss=loss, metrics=metrics)

but at training time TensorFlow throw me the following error:

ValueError: No gradients provided for any variable:

like image 304
robertspierre Avatar asked Dec 11 '19 19:12

robertspierre


People also ask

Does TensorFlow have gradients for each variable?

Tensorflow 2.0: No gradients provided for any variable but only when tf.math.square AND tf.function is used? · Issue #27949 · tensorflow/tensorflow · GitHub

How to use loss functions in TensorFlow?

In Tensorflow, these loss functions are already included, and we can just call them as shown below. model.compile (loss = ‘binary_crossentropy’, optimizer = ‘adam’, metrics = [‘accuracy’]) 2. Loss function as an object from tensorflow.keras.losses import mean_squared_error model.compile (loss = mean_squared_error, optimizer=’sgd’)

What is myhuberloss in TensorFlow?

MyHuberLoss is the class name. After the class name, we inherit the parent class ‘Loss’ from tensorflow.keras.losses. So MyHuberLoss inherits as Loss. This allows us to use MyHuberLoss as a loss function.


1 Answers

The problem is in your loss function (obviously). Particularly, the following operation.

y_pred = tf.cast(y_pred > 0.5, tf.float32)

This is not a differentiable operation. Which results in Gradients being None. Change your loss function to the following and it will work.

def dice_loss(y_true, y_pred):
    print("[dice_loss] y_pred=",y_pred,"y_true=",y_true)
    y_true = tf.cast(y_true, tf.float32)
    numerator = 2 * tf.reduce_sum(y_true * y_pred)
    denominator = tf.reduce_sum(y_true + y_pred)

    return 1 - numerator / denominator
like image 139
thushv89 Avatar answered Sep 29 '22 19:09

thushv89