Bigger batch size reduce training time

Question

i'm using CNN for image classification; I do data augmentation with keras ImageDataGenerator

I think i'm missing something.

A ///  train =model.fit_generator(image_gen.flow(train_X, train_label, batch_size=64),epochs=100,verbose=1,validation_data=(valid_X, valid_label),class_weight=class_weights,callbacks=[metrics],steps_per_epoch=len(train_X)/64)  # 1 epoch =20 secondes

B /// train =model.fit_generator(image_gen.flow(train_X, train_label, batch_size=15),epochs=100,verbose=1,validation_data=(valid_X, valid_label),class_weight=class_weights,callbacks=[metrics],steps_per_epoch=len(train_X)/15)  # 1 epoch = 60 secondes

C /// train =model.fit_generator(image_gen.flow(train_X, train_label, batch_size=256),epochs=100,verbose=1,validation_data=(valid_X, valid_label),class_weight=class_weights,callbacks=[metrics],steps_per_epoch=len(train_X)/256)   # 1 epoch =345secondes

In situation A I use a batch size of 64, I need 20 sec per epoch. Situation B with batch size of 15 i need 60sec per epoch. Situation C with batch size 256 need 345 seconde per epoch.

What I understand:

-batch size = number of image used for weight update. If i got 100 image, with a batch size of 10, weight will be update 10 time at each epoch. Am I right?

Steps_per_epoch is the number of image generated by Keras DataAugmenteur. Since I assign the value length(Train_X)/batch_size, it's should mean, batch_size data are used before weight update. Am i right?

If my two affirmation are true, it's should mean reduccing batch size will increase time for 1 epoch since there will be more weight update.

Why my lower epoch time is when I use a batch size of 64? Why I got a big epoch time when using batch size=256?

Let me know if you know more information or reformulation

EDIT: I dont understand why but when i set batch size =256, i have a total number of steps (batches of samples) per epoch of 256 when it should be len(Train_X)/256 (=58)

Answer 1

the time of update weight almost don't count in this situation

you can check if the calculate need to transfer thing between memory and disk

when use 256 size

you can setup Concurrent tasks if you want to use add batch size to reduce time!