What is the most efficient way to compute a Kronecker Product in TensorFlow?

Question

I am interested in implementing this paper on Kronecker Recurrent Units in TensorFlow.

This involves the computation of a Kronecker Product. TensorFlow does not have an operation for Kronecker Products. I am looking for an efficient and robust way to compute this.

Does this exist, or would I need to define a TensorFlow op manually?

Answer 1

If you will read the math definition of conv2d_transpose and see what Kronecker product calculates, you will see that with the appropriate size of stides for conv2d_tranpose (width, height of the second matrix), it does the same thing.

Moreover you even have batching of Kronecker's product out of the box with conv2d_transpose.

---

Here is an example of you which calculates the Kronecker's product for matrices from wiki.

import tensorflow as tf
a = [[1, 2], [3, 4]]
b = [[0, 5], [6, 7]]

i, k, s = len(a), len(b), len(b)
o = s * (i - 1) + k

a_tf  = tf.reshape(tf.constant(a, dtype=tf.float32), [1, i, i, 1])
b_tf = tf.reshape(tf.constant(b, dtype=tf.float32), [k, k, 1, 1])

res = tf.squeeze(tf.nn.conv2d_transpose(a_tf, b_tf, (1, o, o, 1), [1, s, s, 1], "VALID"))

with tf.Session() as sess:
    print sess.run(res)

Notice that in the case of a non-square matrix, you will need to calulcate more dimensions in the lines:

i, k, s = len(a), len(b), len(b)
o = s * (i - 1) + k

and use them properly as your strides/outputs arguments.