tensorflow placeholder shape [None] [None,1] difference

Question

I am new to tensorflow. I experimented a DQN algorithm with a section involving

a = tf.placeholder(tf.int32, shape = [None],name='A')
q = tf.reduce_sum(critic_q * tf.one_hot(a,n_outputs),axis=1,keepdims=True,name='Q')#Q value for chosen action

y = tf.placeholder(tf.float32, shape = [None],name='Y')

learning_rate = 1e-4
cost = tf.reduce_mean(tf.square(y-q))#mean squared error
global_step = tf.Variable(0,trainable=False,name='global_step')
optimizer = tf.train.AdamOptimizer(learning_rate)
training_op = optimizer.minimize(cost,global_step=global_step)

and initialized the input y with y_batch=np.zeros(nbatch). The network hardly trained at all.

Then, I switched to defining y as

y = tf.placeholder(tf.float32, shape = [None,1],name='Y')

and initialized the input with y_batch=np.zeros(nbatch).reshape(-1,1), which worked nicely.

What was happening in the first implementation?

Answer 1

Every tensor has a rank (number of dimensions) and a set of dimensions.

A placeholder with shape [1] is a placeholder with rank 1 and the dimension in position 0 of 1.

A placeholder with shape [None, 1] is a placeholder with rank 2, hence it has 2 dimensions. The first dimension (index 0) has unknown size (it will be resolved at runtime) while the second dimension (index 1) has the known size of 1.

In order to be compatible, tensors must have the same rank a dimensions.

You can read a more complete assessment about the tensors shape here: https://pgaleone.eu/tensorflow/2018/07/28/understanding-tensorflow-tensors-shape-static-dynamic/#tensors-the-basic