How can I add labels to TensorBoard Images?

Question

TensorBoard is a great tool, but can it be more robust? The image below shows the visualization in TensorBoard.

It's called by the following code:

tf.image_summary('images', images, max_images=100)

As the API suggests, the last digit is the "image number", from 0 to 99 in this case since I specified max_images = 100. I would like to ask, if I can append the label of this image into the text? This would be a great functionality to have as it allows users to see in real time the images and their respective labels during training. In the event whereby some images are totally mislabelled, a fix can be implemented. In other words, I would like the corresponding text in the image below to be:

images/image/9/5
images/image/39/6
images/image/31/0
images/image/30/2
where last digit is the label.

Thanks!

Answer 1

I haven't been able to find a way to do this using only tensorflow, so instead I do the following:

1. Create a placeholder for the summary images (e.g. like a (10, 224, 224, 3) for ten summary images).
2. Create the image summary based on that placeholder.
3. During validation (or training, if you like), pull the images and labels for your summary into python using something like session.run([sample_images, sample_labels]).
4. Iterate through the batch and use OpenCV to write the label onto the image using cv2.putText.
5. Run the summary op providing the labeled images for the placeholder.

Answer 2

Here is a small improvement for approach proposed by Vince Gatto. We can use tf.py_func to avoid creating extra placeholders and doing extra session.run.

First, we define these function (you will need opencv-python installed):

import cv2
import tensorflow as tf

def put_text(imgs, texts):
    result = np.empty_like(imgs)
    for i in range(imgs.shape[0]):
        text = texts[i]
        if isinstance(text, bytes):
            text = text.decode()
        # You may need to adjust text size and position and size.
        # If your images are in [0, 255] range replace (0, 0, 1) with (0, 0, 255)
        result[i, :, :, :] = cv2.putText(imgs[i, :, :, :], str(text), (0, 30), cv2.FONT_HERSHEY_COMPLEX, 1, (0, 0, 1), 2)
    return result

def tf_put_text(imgs, texts):
    return tf.py_func(put_text, [imgs, texts], Tout=imgs.dtype)

Now we can use tf_put_text to print labels on top images just before feeding them to image summary:

annotated_images = tf_put_text(images, labels)
tf.summary.image('annotated_images', annotated_images, 4)