TensorFlow error: logits and labels must be same size

Question

I've been trying to learn TensorFlow by implementing ApproximatelyAlexNet based on various examples on the internet. Basically extending the AlexNet example here to take 224x224 RGB images (rather than 28x28 grayscale images), and adding a couple more layers, changing kernel sizes, strides, etc, per other AlexNet implementations I've found online.

Have worked through a number of mismatched shape type errors, but this one has me stumped:

tensorflow.python.framework.errors.InvalidArgumentError: logits and labels must be same size: logits_size=dim { size: 49 } dim { size: 10 } labels_size=dim { size: 1 } dim { size: 10 }
     [[Node: SoftmaxCrossEntropyWithLogits = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"](Softmax, _recv_Placeholder_1_0/_13)]]
     [[Node: gradients/Mean_grad/range_1/_17 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_457_gradients/Mean_grad/range_1", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

The 49 dimension is particularly puzzling. For debugging, my batch size is currently 1, if I increase it to 2 then the 49 becomes 98.

If I log the shape of the x and y that I pass to

sess.run(optimizer, feed_dict={x: batchImages, y: batchLabels, keepProb: P_DROPOUT})

I get

x shape: (1, 150528)
y shape: (1, 10)

Which is as expected: 150528 = 224 * 224 RGB pixels, and a one-hot vector representing my labels.

Would appreciate any help in figuring this out!

Update: code exhibiting the fault here:

https://gist.github.com/j4m3z0r/e70096d0f7bd4bd24c42

Answer 1

Thanks for sharing your code as a Gist. There are two changes that are necessary to make the shapes agree:

1. The line:

   fc1 = tf.reshape(pool5, [-1, wd1Shape[0]])
   

   ...is responsible for the erroneous 49 in the batch dimension. The input is 1 x 7 x 7 x 256, and it is reshaped to be 49 x 256, because wd1Shape[0] is 256. One possible replacement is the following:

   pool5Shape = pool5.get_shape().as_list()
   fc1 = tf.reshape(pool5, [-1, pool5Shape[1] * pool5Shape[2] * pool5Shape[3]])
   

   ...which will give fc1 the shape 1 x 12544.

2. After making this change, the size of the 'wd1' weight matrix (256 x 4096) doesn't match the number of nodes in fc1. You could change the definition of this matrix as follows:

       'wd1': tf.Variable(tf.random_normal([12544, 4096])),
   

   ...although you may want to modify the other weights, or perform additional pooling to reduce the size of this matrix.

Answer 2

I had a similar issue when using model.fit(..). Turns out my output_size was defined as 2 while using "binary_crossentropy" as the loss function, when it should have been defined as 1.