ValueError when performing matmul with Tensorflow

Question

I'm a total beginner to TensorFlow, and I'm trying to multiply two matrices together, but I keep getting an exception that says:

ValueError: Shapes TensorShape([Dimension(2)]) and TensorShape([Dimension(None), Dimension(None)]) must have the same rank

Here's minimal example code:

data = np.array([0.1, 0.2])
x = tf.placeholder("float", shape=[2])
T1 = tf.Variable(tf.ones([2,2]))
l1 = tf.matmul(T1, x)
init = tf.initialize_all_variables()

with tf.Session() as sess:
    sess.run(init)
    sess.run(feed_dict={x: data}

Confusingly, the following very similar code works fine:

data = np.array([0.1, 0.2])
x = tf.placeholder("float", shape=[2])
T1 = tf.Variable(tf.ones([2,2]))
init = tf.initialize_all_variables()

with tf.Session() as sess:
    sess.run(init)
    sess.run(T1*x, feed_dict={x: data}

Can anyone point to what the issue is? I must be missing something obvious here..

Answer 1

The tf.matmul() op requires that both of its inputs are matrices (i.e. 2-D tensors)^*, and doesn't perform any automatic conversion. Your T1 variable is a matrix, but your x placeholder is a length-2 vector (i.e. a 1-D tensor), which is the source of the error.

By contrast, the * operator (an alias for tf.multiply()) is a broadcasting element-wise multiplication. It will convert the vector argument to a matrix by following NumPy broadcasting rules.

To make your matrix multiplication work, you can either require that x is a matrix:

data = np.array([[0.1], [0.2]])
x = tf.placeholder(tf.float32, shape=[2, 1])
T1 = tf.Variable(tf.ones([2, 2]))
l1 = tf.matmul(T1, x)
init = tf.initialize_all_variables()

with tf.Session() as sess:
    sess.run(init)
    sess.run(l1, feed_dict={x: data})

...or you could use the tf.expand_dims() op to convert the vector to a matrix:

data = np.array([0.1, 0.2])
x = tf.placeholder(tf.float32, shape=[2])
T1 = tf.Variable(tf.ones([2, 2]))
l1 = tf.matmul(T1, tf.expand_dims(x, 1))
init = tf.initialize_all_variables()

with tf.Session() as sess:
    # ...

^* This was true when I posted the answer at first, but now tf.matmul() also supports batched matrix multiplications. This requires both arguments to have at least 2 dimensions. See the documentation for more details.