How to choose the number of units for the Dense layer in the Convoluted neural network for a Image classification problem?

Question

from keras import layers
from keras import models
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',
input_shape=(150, 150, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

While reading the code for a binary classification problem on classifying images as either cats or dogs, in the Dense layer, they used 512 units. How did they come up with that? Is there a formula to get the number of units in the Dense layer. Usually if there are many features, we choose large number of units in the Dense layer.But here how do we identify the features?I know that the output Dense layer has one unit as its a binary classification problem so the out put will either be 0 or 1 by sigmoid function.

Answer 1

My experience with CNNs is to start out with a simple model initially and evaluate its performance. If you achieve a satisfactory level of training and validation accuracy stop there. If not try adjusting hyper parameters like learning rate to achieve better performance before adding more complexity to your model. I have found using an adjustable learning rate to be helpful in improving model performance. Use the Keras callback ReduceLROnPlateau for this purpose. Set it to monitor validation accuracy and reduce the learning rate if it fails to improve after a specified number of epochs. Documentation is here. Also use the Keras callback ModelCheckpoint to save the model with the lowest validation loss. Documentation is here. If these methods do not achieve the desired level of training accuracy, then you may want to increase the model complexity by adding more nodes to the dense layer or adding additional dense layers. If your model had high training accuracy but poor validation accuracy your model may be over fitting. In this case add a dropout layer. Documentation is here. The issue with adding more complexity to your model is the tendency for it to over fit. So if you increase the nodes in the dense layer or add additional dense layers and have poor validation accuracy you will have to add dropout. In addition you may want to consider alternate approaches to control over fitting like regularizers. Documentation for that is here. For your specific example I think you have more nodes in the dense layer then is needed. Try something like 64 nodes to begin with.