Menu
keras has the ability to export some of it's training data in a tensorboard comaptible format using keras.callbacks.TensorBoard
However, it doesn't support the embedding visualisation in tensorboard.
Is there a way around this?
847
In order to visualize the embeddings, we select PROJECTOR from the dropdown menu on the top right of the TensorBoard dashboard: Image by author.
Using TensorBoard with deep learning frameworksYou can also use it with other frameworks such as Keras, PyTorch and XGBoost, just to mention a few.
To visualize the word embedding, we are going to use common dimensionality reduction techniques such as PCA and t-SNE. To map the words into their vector representations in embedding space, the pre-trained word embedding GloVe will be implemented.
The Embedding Projector offers three commonly used methods of data dimensionality reduction, which allow easier visualization of complex data: PCA, t-SNE and custom linear projections. PCA is often effective at exploring the internal structure of the embeddings, revealing the most influential dimensions in the data.
Found a solution:
import os
import keras
import tensorflow
ROOT_DIR = '/tmp/tfboard'
os.makedirs(ROOT_DIR, exist_ok=True)
OUTPUT_MODEL_FILE_NAME = os.path.join(ROOT_DIR,'tf.ckpt')
# get the keras model
model = get_model()
# get the tensor name from the embedding layer
tensor_name = next(filter(lambda x: x.name == 'embedding', model.layers)).W.name
# the vocabulary
metadata_file_name = os.path.join(ROOT_DIR,tensor_name)
embedding_df = get_embedding()
embedding_df.to_csv(metadata_file_name, header=False, columns=[])
saver = tensorflow.train.Saver()
saver.save(keras.backend.get_session(), OUTPUT_MODEL_FILE_NAME)
summary_writer = tensorflow.train.SummaryWriter(ROOT_DIR)
config = tensorflow.contrib.tensorboard.plugins.projector.ProjectorConfig()
embedding = config.embeddings.add()
embedding.tensor_name = tensor_name
embedding.metadata_path = metadata_file_name
tensorflow.contrib.tensorboard.plugins.projector.visualize_embeddings(summary_writer, config)
There is pull request with this functionality - https://github.com/fchollet/keras/pull/5247 callback is extended to create visualization for specific embedding layers.
41
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
