Sparse matrices in python tensorflow text classification

Question

I've been trying to implement a text classification routine using the tensorflow package in python. I already had a successful perceptron version working in the scikit-learn enviroment but scikit-learn does not have multilayer neural networks (except for some mythical version 0.18 that I can't seem to find/install anywhere).

I thought it was best to try something simpler in tensorflow first, to learn how the package works and what it can and cannot do, so I went with nearest neighbors. So far so good, except I just can't find a way to feed a sparse version of the vocabulary matrix (bag-of-words vectorizations of the texts) to a placeholder in tensorflow (in scikit-learn this is no problem at all). Converting the vocabulary matrix to a dense matrix solves the problem but severely slows down the algorithm and clogs up RAM.

Is there any way around this? From what I found on the web it seems tensorflow has very limited support for sparse objects (only certain operations will accept them as input), but I hope I'm wrong.

P.S. Yes, I read this thread and it did not solve my problem. And yes, I know I could stick to the perceptron of the scikit-learn or choose another package, but that's a decision I'll make based on the answers I get here.

Answer 1

With TensorFlow 1.0.1, I can do this:

a = sparse.csr_matrix([[0, 1, 2], [5, 0, 0], [0, 0, 5],
                       [10, 1, 0], [0, 0, 4]])
# w = np.arange(6, dtype=np.float32).reshape([3, 2])
a = a.tocoo()

a_ = tf.sparse_placeholder('float32')
w_ = tf.Variable(tf.random_normal([3, 2], stddev=1.0))

a_sum = tf.sparse_reduce_sum(a_, 1)
a_mul = tf.sparse_tensor_dense_matmul(a_, w_)

# Initializing the variables
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    indices = np.array(zip(a.row, a.col), dtype=np.int32)
    values = np.array(a.data, dtype=np.float32)
    shape = np.array(a.shape, dtype=np.int32)

    print sess.run(a_mul, feed_dict={a_: tf.SparseTensorValue(indices, values, shape)})
    w = sess.run(w_)
    print np.dot(a.todense(), w)

You can find the code from API page: sparse placeholder. After the first layer, other layers of neural network will be dense matrix.