TensorBoard Embedding Example?

Question

I'm looking for a tensorboard embedding example, with iris data for example like the embedding projector http://projector.tensorflow.org/

But unfortunately i couldn't find one. Just a little bit information about how to do it in https://www.tensorflow.org/how_tos/embedding_viz/

Does someone knows a basic tutorial for this functionality?

Basics:

1) Setup a 2D tensor variable(s) that holds your embedding(s).

embedding_var = tf.Variable(....) 

2) Periodically save your embeddings in a LOG_DIR.

3) Associate metadata with your embedding.

Answer 1

I've used FastText's pre-trained word vectors with TensorBoard.

import os import tensorflow as tf import numpy as np import fasttext from tensorflow.contrib.tensorboard.plugins import projector  # load model word2vec = fasttext.load_model('wiki.en.bin')  # create a list of vectors embedding = np.empty((len(word2vec.words), word2vec.dim), dtype=np.float32) for i, word in enumerate(word2vec.words):     embedding[i] = word2vec[word]  # setup a TensorFlow session tf.reset_default_graph() sess = tf.InteractiveSession() X = tf.Variable([0.0], name='embedding') place = tf.placeholder(tf.float32, shape=embedding.shape) set_x = tf.assign(X, place, validate_shape=False) sess.run(tf.global_variables_initializer()) sess.run(set_x, feed_dict={place: embedding})  # write labels with open('log/metadata.tsv', 'w') as f:     for word in word2vec.words:         f.write(word + '\n')  # create a TensorFlow summary writer summary_writer = tf.summary.FileWriter('log', sess.graph) config = projector.ProjectorConfig() embedding_conf = config.embeddings.add() embedding_conf.tensor_name = 'embedding:0' embedding_conf.metadata_path = os.path.join('log', 'metadata.tsv') projector.visualize_embeddings(summary_writer, config)  # save the model saver = tf.train.Saver() saver.save(sess, os.path.join('log', "model.ckpt")) 

Then run this command in your terminal:

tensorboard --logdir=log 

Answer 2

It sounds like you want to get the Visualization section with t-SNE running on TensorBoard. As you've described, the API of Tensorflow has only provided the bare essential commands in the how-to document.

I’ve uploaded my working solution with the MNIST dataset to my GitHub repo.

Yes, it is broken down into three general steps:

1. Create metadata for each dimension.
2. Associate images with each dimension.
3. Load the data into TensorFlow and save the embeddings in a LOG_DIR.

Only generic details are inculded with the TensorFlow r0.12 release. There is no full code example that I’m aware of within the official source code.

I found that there were two tasks involved that were not documented in the how to.

1. Preparing the data from the source
2. Loading the data into a tf.Variable

While TensorFlow is designed for the use of GPUs, in this situation I opted to generate the t-SNE visualization with the CPU as the process took up more memory than my MacBookPro GPU has access to. API access to the MNIST dataset is included with TensorFlow, so I used that. The MNIST data comes as a structured a numpy array. Using the tf.stack function enables this dataset to be stacked into a list of tensors which can be embedded into a visualization. The following code contains is how I extracted the data and setup the TensorFlow embedding variable.

with tf.device("/cpu:0"):     embedding = tf.Variable(tf.stack(mnist.test.images[:FLAGS.max_steps], axis=0), trainable=False, name='embedding') 

Creating the metadata file was perfomed with the slicing of a numpy array.

def save_metadata(file):     with open(file, 'w') as f:         for i in range(FLAGS.max_steps):             c = np.nonzero(mnist.test.labels[::1])[1:][0][i]             f.write('{}\n'.format(c)) 

Having an image file to associate with is as described in the how-to. I've uploaded a png file of the first 10,000 MNIST images to my GitHub.

So far TensorFlow works beautifully for me, it’s computationaly quick, well documented and the API appears to be functionally complete for anything I am about to do for the moment. I look forward to generating some more visualizations with custom datasets over the coming year. This post was edited from my blog. Best of luck to you, please let me know how it goes. :)