CNN Overfitting (with output and code)

Question

I have a dataset containing 20000 black and white images of 2 classes I want to classify (the images kinda look like weather forecast or stock market charts, so I can't use pretrained networks). The dataset has been split into 18000 images for training and 2000 images for testing purpose. I am training it using Keras convolutional neural networks. I have got 96% accuracy with training set but results obtained with test set are not good (it gets stuck at 82-83% after 50 epochs). I think it may be because of overfitting. Please can you suggest me something to solve the problem. I leave final output and code for you to see.

281/281 [==============================] - 132s - loss: 0.1024 - acc: 0.9612 - val_loss: 0.5836 - val_acc: 0.8210

from keras.layers.normalization import BatchNormalization
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, ZeroPadding2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
from keras.optimizers import SGD, RMSprop, Adam
from keras.callbacks import ModelCheckpoint 


filepath="weights/weights-improvement-{epoch:02d}-{val_acc:.4f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max')
callbacks_list = [checkpoint]


img_width, img_height = 100, 1296

train_data_dir = 'dataset/train'
validation_data_dir = 'dataset/validation'
nb_train_samples = 18000
nb_validation_samples = 2000
epochs = 100
batch_size = 64

input_shape = (img_width, img_height, 1)

model = Sequential()

model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(ZeroPadding2D((1, 1)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.1))

model.add(ZeroPadding2D((1, 1)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))

model.add(ZeroPadding2D((1, 1)))
model.add(Conv2D(128, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.3))

model.add(ZeroPadding2D((1, 1)))
model.add(Conv2D(256, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.4))

model.add(ZeroPadding2D((1, 1)))
model.add(Conv2D(512, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))

model.add(Flatten())

model.add(Dense(256))
model.add(Activation('relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(256))
model.add(Activation('relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
               optimizer=Adam(lr=0.001),
              metrics=['accuracy'])

model.save('model.h5')

train_datagen = ImageDataGenerator(
    rescale=None,
)

test_datagen = ImageDataGenerator(rescale=None)

train_generator = train_datagen.flow_from_directory(
    train_data_dir, color_mode='grayscale',
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir, color_mode='grayscale',
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size,callbacks=callbacks_list)

Answer 1

You have a huge number of parameters in your network- due to several large convolutional layers and two large fully connected layers. This may lead to overfitting. I would suggest to decrease their sizes, and maybe get rid of several of the layers completely.

Answer 2

Since you have an overfitting problem, why don't you try regularization? You can each layer can attach to a regularizer. I normally use a weight regularizer, which is now called kernel regularizer. You can find the Keras regularizer documentation at https://keras.io/regularizers/

Increase the drop out rate might also help, but try to use regularization first. Also having more training data also helps reduce overfitting, but it is not always possible to come up with more training data.