Menu
I am building a plain vanilla FNN and want to evaluate my model after training. I was wondering what impact the batch_size has when evaluating the model on a test set. Of course it is relevant for training as it determines the number of samples to be fed to the network before computing the next gradient. It is also clear that it can be needed when predicting values for a (statefull) RNN. But it is not clear to me why it is needed when evaluating the model especially a FNN. Furthermore, I get slightly different values when I evaluate the model on the same test set but with different batch sizes. Consider the following toy example:
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.optimizers import SGD
# function to be learned
def f(x):
return x[0] + x[1] + x[2]
# sample training and test points on a rectangular grid
x_train = np.random.uniform(low = -10, high = 10, size = (50,3))
y_train = np.apply_along_axis(f, 1, x_train).reshape(-1,1)
x_test = np.random.uniform(low = -10, high = 10, size = (50,3))
y_test = np.apply_along_axis(f, 1, x_test).reshape(-1,1)
model = Sequential()
model.add(Dense(20, input_dim = 3, activation = 'tanh'))
model.add(Dense(1))
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='mse',
optimizer=sgd)
model.fit(x_train, y_train, batch_size = 10, epochs = 30, verbose = 0)
model.evaluate(x_test, y_test, batch_size = 10)
model.evaluate(x_test, y_test, batch_size = 20)
model.evaluate(x_test, y_test, batch_size = 30)
model.evaluate(x_test, y_test, batch_size = 40)
model.evaluate(x_test, y_test, batch_size = 50)
The values are very similar but nevertheless different. Where does this come from? Shouldn't the following be always true?
from sklear.metrics import mean_squared_error as mse
0 == model.evaluate(x_test, y_test) - mse(model.predict(x_test), y_test)
422
The batch size is a number of samples processed before the model is updated. The number of epochs is the number of complete passes through the training dataset. The size of a batch must be more than or equal to one and less than or equal to the number of samples in the training dataset.
Batch size is a term used in machine learning and refers to the number of training examples utilized in one iteration. The batch size can be one of three options: batch mode: where the batch size is equal to the total dataset thus making the iteration and epoch values equivalent.
Put simply, the batch size is the number of samples that will be passed through to the network at one time. Note that a batch is also commonly referred to as a mini-batch. The batch size is the number of samples that are passed to the network at once.
Batch Size is the number of samples we send to the model at a time. In this example, we have batch size = 2 but you can take it 4, 8,16, 32, 64 etc depends on the memory (basically in 2's power)
No, they don't have to be the same. If you combine floating point math with parallelism, you don't get reproducible results as then (a + b) + c is not the same as a + (b + c).
The evaluate function of Model has a batch size just in order to speed-up evaluation, as the network can process multiple samples at a time, and with a GPU this makes evaluation much faster. I think the only way to reduce the effect of this would be to set batch_size to one.
76
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
