Efficient element-wise multiplication of a matrix and a vector in TensorFlow

Question

What would be the most efficient way to multiply (element-wise) a 2D tensor (matrix):

x11 x12 .. x1N ... xM1 xM2 .. xMN 

by a vertical vector:

w1 ... wN 

to obtain a new matrix:

x11*w1 x12*w2 ... x1N*wN ... xM1*w1 xM2*w2 ... xMN*wN 

To give some context, we have M data samples in a batch that can be processed in parallel, and each N-element sample must be multiplied by weights w stored in a variable to eventually pick the largest Xij*wj for each row i.

Answer 1

The simplest code to do this relies on the broadcasting behavior of tf.multiply()^*, which is based on numpy's broadcasting behavior:

x = tf.constant(5.0, shape=[5, 6]) w = tf.constant([0.0, 1.0, 2.0, 3.0, 4.0, 5.0]) xw = tf.multiply(x, w) max_in_rows = tf.reduce_max(xw, 1)  sess = tf.Session() print sess.run(xw) # ==> [[0.0, 5.0, 10.0, 15.0, 20.0, 25.0], #      [0.0, 5.0, 10.0, 15.0, 20.0, 25.0], #      [0.0, 5.0, 10.0, 15.0, 20.0, 25.0], #      [0.0, 5.0, 10.0, 15.0, 20.0, 25.0], #      [0.0, 5.0, 10.0, 15.0, 20.0, 25.0]]  print sess.run(max_in_rows) # ==> [25.0, 25.0, 25.0, 25.0, 25.0] 

^* In older versions of TensorFlow, tf.multiply() was called tf.mul(). You can also use the * operator (i.e. xw = x * w) to perform the same operation.