Is incremental learning possible with Tensorflow?

Question

I'm trying to train a Tensorflow model with a very large dataset (much larger than my memory).

To fully utilize all the available training data, I'm thinking about separating them into several small "shards" and train on one shard at a time.

After a little research, I found this method is often referred to as "incremental learning". And based on this Wiki page, NOT all algorithms support incremental learning.

I'm building my model using tf.keras.Model. In this case, is incremental learning possible?

Answer 1

Tensorflow & Keras models support incremental learning by default - in fact, we routinely use incremental learning in cases of transfer learning, among others. You just fit different portions of your data to the model sequentially - you can even save your model, and then load it and continue training with the same or different parts of your data.

For more info see:

• Incremental learning in keras (SO thread)
• Transfer learning & fine-tuning (Keras dosumentation)
• Online/Incremental Learning with Keras and Creme
• Can I train a neural network incrementally given new daily data? (AI SE thread)