Menu
In a CNN for binary classification of images, should the shape of output be (number of images, 1) or (number of images, 2)? Specifically, here are 2 kinds of last layer in a CNN:
keras.layers.Dense(2, activation = 'softmax')(previousLayer)
or
keras.layers.Dense(1, activation = 'softmax')(previousLayer)
In the first case, for every image there are 2 output values (probability of belonging to group 1 and probability of belonging to group 2). In the second case, each image has only 1 output value, which is its label (0 or 1, label=1 means it belongs to group 1).
Which one is correct? Is there intrinsic difference? I don't want to recognize any object in those images, just divide them into 2 groups.
Thanks a lot!
637
Dense Layer is simple layer of neurons in which each neuron receives input from all the neurons of previous layer, thus called as dense. Dense Layer is used to classify image based on output from convolutional layers. Working of single neuron. A layer contains multiple number of such neurons.
As known, the main difference between the Convolutional layer and the Dense layer is that Convolutional Layer uses fewer parameters by forcing input values to share the parameters. The Dense Layer uses a linear operation meaning every output is formed by the function based on every input.
CNN is composed of 2 batch-norm layers, 3 convolutional layers, 2 max-pooling layers, 3 hidden dense layers, 4 dropout layers (used only for the training) and one output layer.
The Flatten layer converts the 28x28x32 output of the convolutional layer into a single one-dimensional vector, that can be used as input for a dense layer. The last dense layer has the most parameters. This layer connects every single output 'pixel' from the convolutional layer to the 10 output classes.
What is really the difference between a Dense Layer and an Output Layer in a CNN also in a CNN with this kind of architecture may one say the Fullyconnected Layer = Dense Layer + Output Layer / Fullyconnected Layer = Dense Layer alone. The convolutional part is used as a dimension reduction technique to map the input vector X to a smaller one.
The last Dense layer of CNN model uses “softmax” activation for processing the output with number of classes = number of neurons for final output layer. tf.keras.layers.Dense (6, activation=’softmax’) The Multi Class uses “categorical_crossentropy” loss function for calculation of loss value.
In a CNN for binary classification of images, should the shape of output be (number of images, 1) or (number of images, 2)? Specifically, here are 2 kinds of last layer in a CNN: In the first case, for every image there are 2 output values (probability of belonging to group 1 and probability of belonging to group 2).
The Dense layers are the ones that are mostly used for the output layers. The activation used is the ‘Softmax’ which gives a probability for each class and they sum up totally to 1. The model will make it’s prediction based on the class with highest probability.
This first one is the correct solution:
keras.layers.Dense(2, activation = 'softmax')(previousLayer)
Usually, we use the softmax activation function to do classification tasks, and the output width will be the number of the categories. This means that if you want to classify one object into three categories with the labels A,B, or C, you would need to make the Dense layer generate an output with a shape of (None, 3). Then you can use the cross_entropyloss function to calculate the LOSS, automatically calculate the gradient, and do the back-propagation process.
If you want to only generate one value with the Dense layer, that means you get a tensor with a shape of (None, 1) - so it produces a single numeric value, like a regression task. You are using the value of the output to represent the category. The answer is correct, but does not perform like the general solution of the classification task.
121
The difference is if the class probabilities are independent of each other (multi-label classification) or not.
When there are 2 classes and you generally have P(c=1) + P(c=0) = 1 then
keras.layers.Dense(2, activation = 'softmax')
keras.layers.Dense(1, activation = 'sigmoid')
both are correct in terms of class probabilities. The only difference being how you supply the labels during training. But
keras.layers.Dense(2, activation = 'sigmoid')
is incorrect in that context. However, it is correct implementation if you have P(c=1) + P(c=0) != 1. This is the case for multi-label classification where an instance may belong to more than one correct class.
33
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
