Accuracy from callback and progress bar in Keras doesnt match

Question

I'm trying to learn Keras and are using LSTM for a classification problem. I want to be able to plot the accuracy and loss and update the plot during training. For that I'm using the callback function. For some reason the accuracy and loss I receive in the callbacks does not match with the accuracy and loss printed by the fit function.

Here are the relevant lines of my code:

class PlotCallbacks(Callback):
    def on_batch_end(self, batch, logs={}):
        print(logs)
        return

# Create the model
model = Sequential()
model.add(Embedding(top_words, embedding_vector_length,input_length=max_conv_length))
model.add(LSTM(300))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, callbacks=[PlotCallbacks()], nb_epoch=1, batch_size=3, verbose=1)

When running the program I get this output (first row of each batch is printed by callback, second is printed by model.fit):

    Epoch 1/1
{'acc': 0.0, 'loss': 1.1038421, 'batch': 0, 'size': 3}
 3/25 [==>...........................] - ETA: 27s - loss: 1.1038 - acc: 0.0000e+00   

 {'acc': 1.0, 'loss': 1.0622898, 'batch': 1, 'size': 3}
 6/25 [======>.......................] - ETA: 19s - loss: 1.0831 - acc: 0.5000       

 {'acc': 1.0, 'loss': 0.91526389, 'batch': 2, 'size': 3}
 9/25 [=========>....................] - ETA: 13s - loss: 1.0271 - acc: 0.6667   

 {'acc': 1.0, 'loss': 0.36570337, 'batch': 3, 'size': 3}
12/25 [=============>................] - ETA: 11s - loss: 0.8618 - acc: 0.7500

{'acc': 1.0, 'loss': 0.1433304, 'batch': 4, 'size': 3}
15/25 [=================>............] - ETA: 9s - loss: 0.7181 - acc: 0.8000

{'acc': 1.0, 'loss': 0.041385528, 'batch': 5, 'size': 3}
18/25 [====================>.........] - ETA: 6s - loss: 0.6053 - acc: 0.8333

{'acc': 1.0, 'loss': 0.011424608, 'batch': 6, 'size': 3}
21/25 [========================>.....] - ETA: 3s - loss: 0.5205 - acc: 0.8571

{'acc': 1.0, 'loss': 0.0034991663, 'batch': 7, 'size': 3}
24/25 [===========================>..] - ETA: 1s - loss: 0.4558 - acc: 0.8750

{'acc': 1.0, 'loss': 0.0012318328, 'batch': 8, 'size': 1}
25/25 [==============================] - 26s - loss: 0.4377 - acc: 0.8800    

I have tried to print logs.get('acc'), as well as saved the accuracies to a list in the PlotCallbacks object and print the list, but the problem remains.

Does anyone have a clue what the problem may be?

Thanks

Answer 1

on_batch_end() type callback function gets the accuracy of the batch that just got trained. Whereas the logs printed by keras is the average over all the batches that it has seen in the current epoch. You can easily observe that in your logs.. say in first 2 batches one accuracy was 0.0 and 1.0, which made the overall accuracy over 2 batches seen as 0.5000. here is exactly where the average is calculated.

Also accuracy as a metric is usually reported from epoch to epoch so you can change the callback to on_epoch_end().