Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to improve my CNN ? high and constant validation error

I am working on a problem for predicting a score of how fat cows are, based on images of cows. I applied a CNN to estimate the value which is between 0-5 ( the dataset i have, contains only values between 2.25 and 4 ) I am using 4 CNN layers and 3 Hidden layers.

I actualy have 2 problems : 1/ I got 0.05 training error, but after 3-5 epochs the validation error remains at about 0.33. 2/ The value predicted by my NN are between 2.9 and 3.3 which is too narrow compared with the dataset range. Is it normal ?

How can i improve my model ?

model = tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(512, 424,1)),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2,2),
    tf.keras.layers.Flatten(input_shape=(512, 424)),
    tf.keras.layers.Dense(256, activation=tf.nn.relu),
    tf.keras.layers.Dense(128, activation=tf.nn.relu),
    tf.keras.layers.Dense(64, activation=tf.nn.relu),
    tf.keras.layers.Dense(1, activation='linear')
])

Learning Curve:

Learning Curve

Prediction:

Prediction

like image 886
Wassim Bouatay Avatar asked Jan 01 '23 19:01

Wassim Bouatay


1 Answers

This seems to be the case of Overfitting. You can

  1. Shuffle the Data, by using shuffle=True in cnn_model.fit. Code is shown below:

    history = cnn_model.fit(x = X_train_reshaped, y = y_train, batch_size = 512, epochs = epochs, callbacks=[callback], verbose = 1, validation_data = (X_test_reshaped, y_test), validation_steps = 10, steps_per_epoch=steps_per_epoch, shuffle = True)

  2. Use Early Stopping. Code is shown below

    callback = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=15)

  3. Use Regularization. Code for Regularization is shown below (You can try l1 Regularization or l1_l2 Regularization as well):

from tensorflow.keras.regularizers import l2

Regularizer = l2(0.001)

cnn_model.add(Conv2D(64,3, 3, input_shape = (28,28,1), activation='relu', data_format='channels_last', activity_regularizer=Regularizer, kernel_regularizer=Regularizer))

cnn_model.add(Dense(units = 10, activation = 'sigmoid', activity_regularizer=Regularizer, kernel_regularizer=Regularizer))

  1. You can try using BatchNormalization.

  2. Perform Image Data Augmentation using ImageDataGenerator. Refer this link for more info about that.

  3. If the Pixels are not Normalized, Dividing the Pixel Values with 255 also helps.

  4. Finally, if there still no change, you can try using Pre-Trained Models like ResNet or VGG Net, etc..

like image 64
Tensorflow Support Avatar answered Jan 17 '23 23:01

Tensorflow Support