Rank mismatch: Rank of labels (received 2) should equal rank of logits minus 1 (received 2)

Question

I'm building DNN to predict if the object is present in the image or not. My network has two hidden layers and the last layer looks like this:

  # Output layer
  W_fc2 = weight_variable([2048, 1])
  b_fc2 = bias_variable([1])

  y = tf.matmul(h_fc1, W_fc2) + b_fc2

Then I have placeholder for labels:

y_ = tf.placeholder(tf.float32, [None, 1], 'Output')

I run training in batches (therefore first argument in Output layer shape is None).

I use the following loss function:

cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(
    y[:, :1], y_[:, :1], name='xentropy')
loss = tf.reduce_mean(cross_entropy, name='xentropy_mean')
predict_hand = tf.greater(y, 0.5)
correct_prediction = tf.equal(tf.to_float(predict_hand), y_)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

But in runtime I got the following error:

Rank mismatch: Rank of labels (received 2) should equal rank of logits minus 1 (received 2).

I guess I should reshape labels layer, but not sure what it expects. I looked up in documentation and it says:

logits: Unscaled log probabilities of rank r and shape [d_0, d_1, ..., d_{r-2}, num_classes] and dtype float32 or float64. labels: Tensor of shape [d_0, d_1, ..., d_{r-2}] and dtype int32 or int64. Each entry in labels must be an index in [0, num_classes).

If I have just single class, what my labels should look like (now it is just 0 or 1)? Any help appreciated

Answer 1

From the documentation* for tf.nn.sparse_softmax_cross_entropy_with_logits:

"A common use case is to have logits of shape [batch_size, num_classes] and labels of shape [batch_size]. But higher dimensions are supported."

So I suppose your labels tensor should be of shape [None]. Note that a given tensor with shape [None, 1] or shape [None] will contain the same number of elements.

Example input with concrete dummy values:

>>> logits = np.array([[11, 22], [33, 44], [55, 66]])
>>> labels = np.array([1, 0, 1])

Where there's 3 examples in the mini-batch, the logits for the first example are 11 and 22 and there's 2 classes: 0 and 1.

*https://www.tensorflow.org/versions/r0.11/api_docs/python/nn.html#sparse_softmax_cross_entropy_with_logits