Multiplying a rank 3 tensor with a rank 2 tensor in Tensorflow

Question

Given a rank 3 tensor:

sentence_max_length = 5
batch_size = 3
n_hidden = 10
n_classes = 2
x = tf.constant(np.reshape(np.arange(150),(batch_size,sentence_max_length, n_hidden)), dtype = tf.float32)

And a rank 2 tensor:

W = tf.constant(np.reshape(np.arange(20), (n_hidden, n_classes)), dtype = tf.float32)

And a rank 1 bias tensor:

b = tf.constant(np.reshape(np.arange(5), (n_classes), dtype = tf.float32))

I was wondering how one would the last two axis of x by W such that the resulting vector Z would be of shape (batch_size, max_length, n_classes) though batch_size would not be known during graph creation I've just given it a value here for demonstration purposes

So to clarify:

Z[0] = tf.matmul(x[0,:,:], W) + b

So that the W and b are shared across all the batches. The reason for this is that I am trying to use the output of tf.dynamic_rnn whereby the output is of shape (batch_size, sentence_max_length, n_hidden) and build another layer ontop of output which has shared weights W and b.

Answer 1

One approach could be ...

import tensorflow as tf
import numpy as np
from tensorflow.python.layers.core import Dense

sentence_max_length = 5
batch_size = 3
n_hidden = 10
n_classes = 2
x = tf.constant(np.reshape(np.arange(150),(batch_size,sentence_max_length, n_hidden)), dtype = tf.float32)

linear_layer = Dense(n_classes, use_bias=True) #required projection value
z = linear_layer(x)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    res = sess.run(z)

res.shape
(3, 5, 2)

Internally, Dense layer creates trainable W & b variables. And, it uses standard_ops.tensordot operation to transform the last dimension to the projected value. For further details, refer to the source code here.