Menu
I am training on 970 samples and validating on 243 samples.
How big should batch size and number of epochs be when fitting a model in Keras to optimize the val_acc? Is there any sort of rule of thumb to use based on data input size?
984
The right number of epochs depends on the inherent perplexity (or complexity) of your dataset. A good rule of thumb is to start with a value that is 3 times the number of columns in your data. If you find that the model is still improving after all epochs complete, try again with a higher value.
The batch size is a number of samples processed before the model is updated. The number of epochs is the number of complete passes through the training dataset. The size of a batch must be more than or equal to one and less than or equal to the number of samples in the training dataset.
In all cases the best results have been obtained with batch sizes m = 32 or smaller, often as small as m = 2 or m = 4. — Revisiting Small Batch Training for Deep Neural Networks, 2018. Nevertheless, the batch size impacts how quickly a model learns and the stability of the learning process.
Since you have a pretty small dataset (~ 1000 samples), you would probably be safe using a batch size of 32, which is pretty standard. It won't make a huge difference for your problem unless you're training on hundreds of thousands or millions of observations.
To answer your questions on Batch Size and Epochs:
In general: Larger batch sizes result in faster progress in training, but don't always converge as fast. Smaller batch sizes train slower, but can converge faster. It's definitely problem dependent.
In general, the models improve with more epochs of training, to a point. They'll start to plateau in accuracy as they converge. Try something like 50 and plot number of epochs (x axis) vs. accuracy (y axis). You'll see where it levels out.
What is the type and/or shape of your data? Are these images, or just tabular data? This is an important detail.
117
Great answers above. Everyone gave good inputs.
Ideally, this is the sequence of the batch sizes that should be used:
{1, 2, 4, 8, 16} - slow { [32, 64],[ 128, 256] }- Good starters [32, 64] - CPU [128, 256] - GPU for more boost If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
