Determining variable rank with unknown shape in Tensorflow

Question

When creating a variable in tensorflow with validate_shape=False, it also ignores the variable rank:

x = tf.placeholder(tf.float32, [None, 10])
v = tf.Variable(tf.ones_like(x), trainable=False, validate_shape=False)
tf.layers.dense(v, 10)

ValueError: Input 0 of layer dense_5 is incompatible with the layer: its rank is undefined, but the layer requires a defined rank.

In this case, while the exact variable shape must be dynamic, I know what its rank is going to be. Is there some way to inform it to tensorflow, so I can use operations that need to know the input rank?

Answer 1

You can do this with tf.reshape():

x = tf.placeholder(tf.float32, [None, 10])
v = tf.Variable(tf.ones_like(x), trainable=False, validate_shape=False)
#tf.layers.dense(v, 10)
tf.layers.dense(tf.reshape(v,[-1,10]),10)

where the -1 allows the return tensor to have shape (?,10); namely the output of the above is:

<tf.Tensor 'dense_10/BiasAdd:0' shape=(?, 10) dtype=float32>

which is what you want. You can verify correct behavior by using a known shape and toggling validate_shape, as in:

x = tf.placeholder(tf.float32, [5, 10])
v = tf.Variable(tf.ones_like(x), trainable=False, validate_shape=True)
tf.layers.dense(v, 10)

...returns the same as:

x = tf.placeholder(tf.float32, [5, 10])
v = tf.Variable(tf.ones_like(x), trainable=False, validate_shape=False)
tf.layers.dense(tf.reshape(v,[5,10]),10)

# returns <tf.Tensor 'dense_8/BiasAdd:0' shape=(5, 10) dtype=float32>