Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Do Keras dropout and GaussianNoise layers apply different instances of noise in every batch?

I am building a denoising autoencoder in Keras. The model I'm using is

input_img = Input(shape=(10,))
encoded = GaussianNoise(0.01)(input_img)
encoded = Dropout(0.1)(encoded)
encoded = Dense(20,activation='relu')(encoded)
decoded = Dense(10, activation='sigmoid')(encoded)
ae = Model(input=input_img, output=decoded)

If I subsequently call

ae.fit(x_train, x_train,
                     nb_epoch=3,
                     batch_size=5,
                     shuffle=True,
                     validation_data=(x_test, x_test))

is there a new instance of the noise created for each batch? In other words, for each epoch above are there different instances of the noise for each of the batches? Or is the noise instance fixed to the same thing for all batches and only changes when the epoch changes? Or worse is there only one noise instance selected for the entire thing?

like image 367
Geordie Avatar asked Jun 26 '16 23:06

Geordie


People also ask

What is Gaussian dropout?

Gaussian noise simply adds random normal values with 0 mean while gaussian dropout simply multiplies random normal values with 1 mean. These operations involve all the elements of the input.


1 Answers

A different instance of the noise is created for each batch during training.

like image 63
Walter Hugo Lopez Pinaya Avatar answered Sep 27 '22 00:09

Walter Hugo Lopez Pinaya