How do I use Batch Normalization during test time in Keras?

Question

I am currently trying to implement a model using Batch Normalization in Keras. I have succesfully implemented it for the training phase.

However, for testing, Batch Normalization calculates the statistics (mean and variance) of the entire population before doing a forward pass through the network (the BN mean and variance are pre-calculated, and then kept static; this is in contrast to the training phase, where the mean and variance are determined by the batch).

My question regarding Keras is:

Assume (X, y) is the entire population. Assume (X_batch, y_batch) is a batch (a subset of that entire population)

If I use the

model.test_on_batch(X_batch, y_batch)

how can I pass on to the batch-normalization layer the mean and variance of the entire population for X and y? Is there any way I can let keras handle this automatically?

Answer 1

how can I pass on to the batch-normalization layer the mean and variance of the entire population for X and y? Is there any way I can let keras handle this automatically?

Keras should just do it (in sufficiently recent versions):

https://github.com/fchollet/keras/issues/81

To double-check, you may want to try batch_size=1 at test/prediction time, and if Keras fails to use the global statistics, you'll probably see very bad results.