Multi-label classification Keras metrics

Question

Which metrics is better for multi-label classification in Keras: accuracy or categorical_accuracy? Obviously the last activation function is sigmoid and as loss function is binary_crossentropy in this case.

Answer 1

I would not use Accuracy for classification tasks with unbalanced classes. Especially for multi-label tasks, you probably have most of your labels to be False. That is, each data point can only have a small set of labels compared to the cardinality of all of the possibile labels. For that reason accuracy is not a good metric, if your model predict all False (sigmoid activation output < 0.5) then you would measure a very high accuracy.

I would analyze either the AUC or recall/precision at each epoch. Alternatively a multi-label task can be seen as a ranking task (like Recommender Systems) and you could evaluate precision@k or recall@k where k are the top predicted labels.

If your Keras back-end is TensorFlow, check out the full list of supported metrics here: https://www.tensorflow.org/api_docs/python/tf/keras/metrics.