How to do multi-class image classification in keras?

Question

Here is what I did. I got the code for dog/cat image classification and I compiled and ran and got 80% accuracy. I added one more class (aeroplane) folder to the train and validation folder. Made changes in the following codes

model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='categorical')

changed binary class_mode to categorical and also loss to categorical_crossentropy. Also changed output layout sigmoid to softmax. Receives the following error.

ValueError: Error when checking target: expected activation_10 to have shape (None, 1) but got array with shape (16, 3)

Do I need to explicity change the training labels to categorical like mentioned below? (I read this from the site multilabel classification using keras)

train_labels = to_categorical(train_labels, num_classes=num_classes) 

I am not sure what happens here. Please help. I am relatively new to deep learning.

model

model = Sequential()

model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='categorical')


validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='categorical')
model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size)

Answer 1

For multi-class classification, the last dense layer must have a number of nodes equal to the number of classes, followed by softmax activation, i.e. the last two layers of your model should be:

model.add(Dense(num_classes))
model.add(Activation('softmax'))

Additionally, your labels (both train and test) must be one-hot encoded; so, assuming that your initial cats and dogs were labeled as integers (0/1), and your new category (airplane) is initially similarly labeled as '2', you should convert them as follows:

train_labels = keras.utils.to_categorical(train_labels, num_classes)
test_labels = keras.utils.to_categorical(test_labels, num_classes)

Finally, on a terminology level, what you are doing is multi-class, and not multi-label classification (I have edited the title of your post) - the last term is used for problems where a sample might belong to more than one categories at the same time.

Answer 2

For the multi-class classification, the size of the last layer of a NN must be equal the number of classes.

F.i. for your problem (3 Classes), the code should look like this:

model = Sequential()

model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(3))
model.add(Activation('softmax'))