Neural Networks: How often is Dropout "filter" updated?

Question

I just have a very general question about Dropout layers. How often is the Dropout "filtering" updated?

For each training example? Or for each mini batch? Or for every epoch?

Thank you very much

Answer 1

Commonly, for each training example.

Source 1: slides taken from Standford CS231n: Convolutional Neural Networks for Visual Recognition:

Source 2: http://www.deeplearningbook.org/ - chapter 7:

Each time we load an example into a minibatch, we randomly sample a different binary mask to apply to all of the input and hidden units in the network. The mask for each unit is sampled independently from all of the others

I would expect that changing for each mini-batch should be fine as well. However, I don't think changing for every epoch is a good idea (especially for large training sets).