How to calculate top5 accuracy in keras?

Question

I want to calculate top5 in imagenet2012 dataset, but i don't know how to do it in keras. fit function just can calculate top 1 accuracy.

Answer 1

If you are just after the topK you could always call tensorflow directly (you don't say which backend you are using).

from keras import backend as K
import tensorflow as tf

top_values, top_indices = K.get_session().run(tf.nn.top_k(_pred_test, k=5))

If you want an accuracy metric you can add it to your model 'top_k_categorical_accuracy'.

model.compile('adam', 'categorical_crossentropy', ['accuracy', 'top_k_categorical_accuracy'])

history = model.fit(X_train, y_train, nb_epoch=3, validation_split=0.2)

Train on 31367 samples, validate on 7842 samples
Epoch 1/3
31367/31367 [==============================] - 6s - loss: 0.0818 - acc: 0.9765 - top_k_categorical_accuracy: 0.9996 - 
...

The default k for this metric is 5 but if you wanted to change that to say 3 you would set up your model like this:

top3_acc = functools.partial(keras.metrics.top_k_categorical_accuracy, k=3)

top3_acc.__name__ = 'top3_acc'

model.compile('adam', 'categorical_crossentropy', ['accuracy', top3_acc])

Answer 2

Frank Wilson's answer is probably the more official answer, but just you can also calculate it like this.

top1 = 0.0
top5 = 0.0    
class_probs = model.predict(x)
for i, l in enumerate(labels):
    class_prob = class_probs[i]
    top_values = (-class_prob).argsort()[:5]
    if top_values[0] == l:
        top1 += 1.0
    if np.isin(np.array([l]), top_values):
        top5 += 1.0

print("top1 acc", top1/len(labels))
print("top1 acc", top5/len(labels))

Answer 3

You can use tf.keras.metrics.TopKCategoricalAccuracy(k). By default k=5.

Documentation is here: https://keras.io/api/metrics/accuracy_metrics/#topkcategoricalaccuracy-class.

Here is code example to use it in model.compile.

import tensorflow as tf

model.compile(optimizer, loss, 
          metrics= [tf.keras.metrics.TopKCategoricalAccuracy(k=5)]
         )