Does EarlyStopping in Keras save the best model?

Question

When using something like:

callbacks = [
    EarlyStopping(patience=15, monitor='val_loss', min_delta=0, mode='min'),
    ModelCheckpoint('best-weights.h5', monitor='val_loss', save_best_only=True, save_weights_only=True)
]

model.fit(..., callbacks=callbacks)

y_pred = model.predict(x_test)

am I doing the prediction with the best weights calculated during training or model is using the last weights (which may not be the best ones)?

So, is the above a safe approach or should I load best-weights.h5 into the model even if the predictions are done right after training?

Answer 1

After the training stops by EarlyStopping callback, the current model may not be the best model with the highest/lowest monitored quantity. As a result a new argument, restore_best_weights, has been introduced in Keras 2.2.3 release for EarlyStopping callback if you would like to restore the best weights:

restore_best_weights: whether to restore model weights from the epoch with the best value of the monitored quantity. If False, the model weights obtained at the last step of training are used.

Answer 2

EarlyStopping callback doesn't save anything on its own (you can double check it looking at its source code https://github.com/keras-team/keras/blob/master/keras/callbacks.py#L458). Thus your code saves the last model that achieved the best result on dev set before the training was stopped by the early stopping callback. I would say that, if you are saving only the best model according to dev, it is not useful to have also an early stopping callback (unless you don't want to save time and your are sure enough you are not going to find any better model if you continue the training)