Using Sparse Tensors to feed a placeholder for a softmax layer in TensorFlow

Question

Has anyone tried using Sparse Tensors for Text Analysis with TensorFlow with success? Everything is ready and I manage to feed feed_dict in tf.Session for a Softmax layer with numpy arrays, but I am unable to feed the dictionary with SparseTensorValues.

I have not found either documentation about using sparse matrices to train a model ( softmax for example ) with Tensor Flow, which is strange, as classes SparseTensor and SparseTensorValues or TensorFlow.sparse_to_dense methods are ready for it, but there is no documentation about how to feed the feed_dict dictionary of values in the session.run(fetches,feed_dict=None) method.

Thanks a lot,

Answer 1

I have found a way of putting sparse images into tensorflow including batch processing if that is of any help.

I create a 4-d sparse matrix in a dictionary where the dimensions are batchSize, xLen, ylen, zLen (where zLen is 3 for colour for example). The following pseudo code is for a batch of 50 32x96 pixel 3-color images. Values are the intensity of each pixel. In the snippet below I show the first 2 pixels of the first batch being initialised...

shape = [50, 32, 96, 3]
indices = [[0, 20, 31, 0],[0, 22, 33, 1], etc...]
values = [12, 24, etc...]
batch = {"indices": indices, "values": values, "shape": shape}

When setting up the computational graph I create a sparse-placeholder of the correct dimensions

images = tf.sparse_placeholder(tf.float32, shape=[None, 32, 96, 3])

'None' is used so I can vary the batch size.

When I first want to use the images, e.g. to feed into a batch convolution, I convert them back to a dense tensor:

images = tf.sparse_tensor_to_dense(batch) 

Then when I am ready to run a session, e.g. for training, I pass the 3 components of the batch into the dictionary so that they will be picked up by the sparse_placeholder:

train_dict = {images: (batch['indices'], batch['values'], batch['shape']), etc...}
sess.run(train_step, feed_dict=train_dict)                

If you are not needing to do batch processing just leave off the first dimension and remove 'none' from the placeholder shape.

I couldn't find any way of passing the images across in batch as an array of sparse matrices. It only worked if I created the 4th dimension. I'd be interested to know of alternatives.

Whilst this doesn't give an exact answer to your question I hope it is of use as I have been struggling with similar issues.