Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Difference in matrix multiplication tensorflow vs numpy

I have a case where matrix multiplication of two matrices with certain dimensions work in numpy, but doesn't work in tensorflow.

x = np.ndarray(shape=(10,20,30), dtype = float)
y = np.ndarray(shape=(30,40), dtype = float)
z = np.matmul(x,y)
print("np shapes: %s x %s = %s" % (np.shape(x), np.shape(y), np.shape(z)))

This works as expected and prints:

np shapes: (10, 20, 30) x (30, 40) = (10, 20, 40)

However in tensorflow when I try to multiply placeholder and variable of the same shapes as the numpy arrays above I get an error

x = tf.placeholder(tf.float32, shape=(10,20,30))
y = tf.Variable(tf.truncated_normal([30,40], name='w'))
print("tf shapes: %s x %s" % (x.get_shape(), y.get_shape()))
tf.matmul(x,y)

Results in

tf shapes: (10, 20, 30) x (30, 40)
InvalidArgumentError: 
Shape must be rank 2 but is rank 3 for 'MatMul_12' 
(op: 'MatMul') with input shapes: [10,20,30], [30,40].

Why does this operation fail?

like image 984
Kuba Avatar asked Feb 12 '17 21:02

Kuba


People also ask

What is the difference between TensorFlow and NumPy?

Tensorflow is a library for artificial intelligence, especially machine learning. Numpy is a library for doing numerical calculations.

Is NumPy matrix multiplication faster?

Matrix multiplications in NumPy are reasonably fast without the need for optimization. However, if every second counts, it is possible to significantly improve performance (even without a GPU).

Is NumPy faster than TensorFlow?

In the second approach I calculate variance via other Tensorflow functions. I tried CPU-only and GPU; numpy is always faster. I used time. time() rather than time.

What matrix multiplication algorithm does NumPy use?

NumPy uses a highly-optimized, carefully-tuned BLAS method for matrix multiplication (see also: ATLAS). The specific function in this case is GEMM (for generic matrix multiplication).


1 Answers

Don't know why tf.matmul does not support this kind of multiplication (may be one of the core developers could provide a meaningful answer).

But if you just want to be able to multiply tensors in this way, take a look at tf.einsum function. It could operate with tensors of arbitrary rank.

like image 107
Dmitriy Danevskiy Avatar answered Oct 18 '22 22:10

Dmitriy Danevskiy