How to repeat an unknown dimension in Keras for both backends

Question

For example (I can do this with Theano without a problem):

# log_var has shape --> (num, )
# Mean has shape --> (?, num)
std_var = T.repeat(T.exp(log_var)[None, :], Mean.shape[0], axis=0)

With TensorFlow I can do this:

std_var = tf.tile(tf.reshape(tf.exp(log_var), [1, -1]), (tf.shape(Mean)[0], 1))

But I don't know how to do the same for Keras, may be like this:

std_var = K.repeat(K.reshape(K.exp(log_var), [1, -1]), Mean.get_shape()[0])

or

std_var = K.repeat_elements(K.exp(log_var), Mean.get_shape()[0], axis=0)

... because Mean has unknown dimension at axis 0.

I need this for a custom layer output:

return K.concatenate([Mean, Std], axis=1)

Answer 1

Keras has an abstraction layer keras.backend which you seem to have found already (you refer to it as K). This layer provides all the functions for both Theano and TensorFlow you will need.

Say your TensorFlow code works, which is

std_var = tf.tile(tf.reshape(tf.exp(log_var), [1, -1]), (tf.shape(Mean)[0], 1))

then you can translate it to the abstract version by writing it like this:

std_var = K.tile(K.reshape(K.exp(log_var), (1, -1)), K.shape(Mean)[0])

Both Theano and TensorFlow support the unknown axis syntax (-1 for unknown axes) so this is not a problem.

On a side note I am not sure whether your TF code is correct though. You reshape to (1, -1), meaning that the dimension of axis 0 will be 1. I think what you rather want to do is to do this:

std_var = K.tile(K.reshape(K.exp(log_var), (-1, num)), K.shape(Mean)[0])