Why is accuracy different between Keras model.fit and model.evaluate?

Question

I am trying to fit a Keras model and use both the history object and evaluate function to see how well the model performs. The code to compute so is below:

optimizer = Adam (lr=learning_rate)
model.compile(loss='categorical_crossentropy', 
              optimizer=optimizer, 
              metrics=['accuracy')
for epoch in range (start_epochs, start_epochs + epochs):
    history = model.fit(X_train, y_train, verbose=0, epochs=1, 
                batch_size=batch_size,
                validation_data=(X_val, y_val))

    print (history.history)
    score = model.evaluate(X_train, y_train, verbose=0)
    print ('Training accuracy', model.metrics_names, score)
    score = model.evaluate(X_val, y_val, verbose=0)
    print ('Validation accuracy', model.metrics_names, score)

To my surprise the accuracy and loss results of the training set differ between history and evaluate. As the results for the validation set are equal it seems some blunder from my side but I cannot find anything. I have given the output for the first four epochs below. I got the same results for metric 'mse': training set differs, test set equal. Anybody any idea?

{'val_loss': [13.354823187591416], 'loss': [2.7036468725265874], 'val_acc': [0.11738484422572477], 'acc': [0.21768202061048531]}
Training accuracy ['loss', 'acc'] [13.265716915499048, 0.1270430906536911]
Validation accuracy ['loss', 'acc'] [13.354821096026349, 0.11738484398216939]

{'val_loss': [11.733116257598105], 'loss': [1.8158155931229045], 'val_acc': [0.26745913783295899], 'acc': [0.34522040671733062]}
Training accuracy ['loss', 'acc'] [11.772184015560292, 0.26721149086656992]
Validation accuracy ['loss', 'acc'] [11.733116155570542, 0.26745913818722139]

{'val_loss': [7.1503656643815061], 'loss': [1.5667824202566349], 'val_acc': [0.26597325444044367], 'acc': [0.44378405117114739]}
Training accuracy ['loss', 'acc'] [7.0615554528994506, 0.26250619121327617]
Validation accuracy ['loss', 'acc'] [7.1503659895943672, 0.26597325408618128]

{'val_loss': [4.2865109046890693], 'loss': [1.4087548087645783], 'val_acc': [0.13893016366866509], 'acc': [0.49232293093422957]}
Training accuracy ['loss', 'acc'] [4.1341019072350802, 0.14338781575775195]
Validation accuracy ['loss', 'acc'] [4.2865103747125541, 0.13893016344725112]

Answer 1

There is nothing to be surprised, the metrics on the training set are just the mean over all batches during training, as the weights are changing with each batch.

Using model.evaluate will keep the model weights fixed and compute loss/accuracy for the whole data you give in. If you want to have the loss/accuracy on the training set, then you have to use model.evaluate and pass the training set to it. The history object does not have the true loss/accuracy on the training set.