Menu
I have a CNN that is learning pretty well on a dataset I created. I added Batch Normalization to this network to try to improve the performances.
But .. when I try to make a prediction on a single image I always end up with the same result (whatever the image). I think it is because I need batches to actually do batch normalization.
So is it possible to do a prediction on a single image with a CNN using BN ? I thought about deleting BN layers once my network is done training, is it the way to go ?
Thank you :)
968
Not good for Recurrent Neural Networks Batch normalization can be applied in between stacks of RNN, where normalization is applied “vertically” i.e. the output of each RNN. But it cannot be applied “horizontally” i.e. between timesteps, as it hurts training because of exploding gradients due to repeated rescaling.
Batch normalization solves a major problem called internal covariate shift. It helps by making the data flowing between intermediate layers of the neural network look, this means you can use a higher learning rate. It has a regularizing effect which means you can often remove dropout.
Unlike batch normalization, group normalization is independent of batch sizes, and also its accuracy is stable in a wide range of batch sizes.
Overall, batch normalized models achieve higher validation and test accuracies on all datasets, which we attribute to its regularizing effect and more stable gradient propagation.
I found the exact answer and the problem I face here : https://r2rt.com/implementing-batch-normalization-in-tensorflow.html in the "Making predictions with the model" it is explained that when using BN, during training time you need to estimate the population mean and population variance on your training set so you don't have to use batch when doing testing (which would be "cheating") :)
93
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
