How can I plot the accuracy as a function of the processing time in Keras?

Question

I've been training a CNN in Keras and plotting the training and validation accuracy as a function of epochs. I was wondering if there is a way of plotting the accuracy as a function of processing time.

Reason being that I want to demonstrate the speed of transfer learning as opposed to retraining a full network. When transfer learning is used, the network takes a similar number of epochs to train, give or take, but each epoch takes far less time (an order of magnitude faster) and I want to capture this graphically.

Here is the code I've been using so far:

history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, verbose=1, validation_data=(X_test, Y_test))

print(history.history.keys())
# summarize history for accuracy
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='lower right')
plt.show()

Answer 1

So it's actually really simple to implement a piece of code to perform your task in Keras. In order to do that - it's good to become familiar with a keras.callback. It makes possible to call a custom functions on:

on_epoch_begin: called at the beginning of every epoch.

on_epoch_end: called at the end of every epoch,

on_batch_begin: called at the beginning of every batch,

on_batch_end: called at the end of every batch,

on_train_begin: called at the beginning of model training,

on_train_end: called at the end of model training.

So now you might implement e.g. a new callback which will :

• register the start of training time on_train_begin,
• at the end of each epoch it will register the actual time of end of computations using on_epoch_end in a dict provided,
• register the end of training on_train_end.

By using the data collected by this callback you could easily present the dependency between time and accuracy in a variety of ways. Of course - it could be easily extended to batch/iterations time periods.