Tensorflow what is the difference between None, -1 and ? when specifying tensor shape?

Question

I have frequently seen the use of the symbols "None", "-1" and "?" to let the program dynamically define a tensor's dimension (such as the batch size), seemingly to the same effect:

x = tf.placeholder(tf.float32, [None, 48, 48, 3], name='InputData')

input = tf.reshape(input, [-1, input_size])

foo("bar", shape=(?,48), dtype=float32)

Are there any functional differences to using any one of these symbols over the other?

Thanks

Answer 1

I think of it this way:

None means an unspecified dimension. So if for example you define a placeholder, you use None to say "this dimension can have any size".
A placeholder can have multiple None dimensions. This simply means multiple dimensions can be different size. Even the whole shape can be None to specify an unknown number of dimensions.

-1 is an instruction to Tensorflow to infer the size of the dimension on its own. In tf.reshape(input, [-1, input_size]), this means "reshape it such that the second dimension is input_size, and the first dimension is whatever is needed to match the total number of elements".
This does not necessarily mean that the dimension is unknown, as it is for None. If your input tensor has a known size of 10 elements, and you reshape to [-1, 2], Tensorflow is able to infer the full shape [5, 2].
-1 is purely for convenience. You could always write down the shape explicitly instead of letting Tensorflow infer it. None on the other hand is necessary for accepting variable-size tensors.
You can only ever have one -1 in a shape. Multiple ones don't make sense because it would be impossible to infer the shape. E.g. if you have 12 elements in a tensor, reshaping to [-1, -1, 2] is undefined -- should we do [3, 2, 2]? [2, 3, 2]? [6, 1, 2]?...

Finally, the question mark is just what Tensorflow uses to mark an "unknown" dimension when printing tensors and/or their shapes. The example you posted would actually produce a syntax error -- you cannot use question marks yourself. Causes of unknown dimensions can of course be placeholders with None dimensions, and often tensors that are defined in terms of placeholders (i.e. the result of some ops applied to them) will have unknown dimensions as well. Also some ops may not specify (parts of) their output shape which can also result in unknowns.

There are likely some more technicalities I missed here, but as a rule of thumb: Use None for placeholders and use -1 for reshape. This should cover the majority of use cases.