How to write a custom loss function in Tensorflow?

Question

I am new to tensorflow. I want to write my own custom loss function. Is there any tutorial about this? For example, the hinge loss or a sum_of_square_loss(though this is already in tf)? Can I do it directly in python or I have to write the cpp code?

Answer 1

We need to write down the loss function. For example, we can use basic mean square error as our loss function for predicted y and target y_:

 loss_mse = 1/n(Sum((y-y_)^2)) 

There are basic functions for tensors like tf.add(x,y), tf.sub(x,y), tf.square(x), tf.reduce_sum(x), etc.

Then we can define our loss function in Tensorflow like:

cost = tf.reduce_mean(tf.square(tf.sub(y,y_))) 

Note: y and y_ are tensors.

Moreover, we can define any other loss functions if we can write down the equations. For some training operators (minimizers), the loss function should satisfy some conditions (smooth, differentiable ...).

In one word, Tensorflow define arrays, constants, variables into tensors, define calculations using tf functions, and use session to run though graph. We can define whatever we like and run it in the end.