Tensorflow - How access loss value for each example in batch?

Question

During training I periodically evaluate my tensorflow network using a modified version of the tensorflow example do_eval() function. My evaluation loss is:

evalLoss = tf.nn.l2_loss(tf.sub(prediction, truthValues_placeholder))

This produces a single scalar loss value. I feed batch truth values in the placeholder via feed_dict.

1. Is that single scalar the loss value for the entire batch?
2. If so, is it the sum of the losses for each example in the batch?
3. If so, how does one access the loss for each example in a batch? I want to see the individual examples for debugging.

Thanks in advance.

Answer 1

As tf.nn.l2_loss(t) just returns sum(t ** 2) / 2,

1. Yes.
2. Yes.
3. You can just name the result of sub() like below.

Then get the eachLoss value by run() or eval() method.

eachLoss = tf.sub(prediction, truthValues_placeholder)
evalLoss = tf.nn.l2_loss(eachLoss)