Keras GaussianNoise layer no effect?

Question

I wanted to add some Gaussian noise to my Images in my CNN with the keras's functional API, but while testing some different stddev values, I noticed that the Gaussian Layer does nothing to the input data. I used the following code for testing:

import tensorflow as tf
import numpy as np
import cv2

stddev = 0.1
image = cv2.imread(<img_path>)
image = (image.astype('float32') - 127.5) / 127.5

input_layer = tf.keras.layers.Input(shape=(128,128,3))
gaus = tf.keras.layers.GaussianNoise(stddev)(input_layer)
model = tf.keras.models.Model(inputs=input_layer, outputs=gaus)

noisy_image = model(image)

print(f'Pixel value at 0,0: {image[0,0]}')
print(f'Pixel value at 0,0: {noisy_image.numpy()[0,0]}')
# Output
# Pixel value at 0,0: [ 0.09803922 -0.30980393 -0.56078434]
# Pixel value at 0,0: [ 0.09803922 -0.30980393 -0.56078434]

It doesn't matter what value I put in for stddev (tried everything from 0.001 to 10000). I expected the values to differ slightly (or heavily when stddev=1000). Am I doing something wrong?

Maybe I should mention that I'm using tensorflow-gpu==2.0.0-rc0 on Windows 10

Answer 1

If you check the docs, it says the layer is only active during training as it's supposed to serve as a regularizer. Looking at the source code confirms this. So it looks like if you will need to make sure the model "knows" that it's in training mode. There are several ways to do this:

• If you use the model.compile/ model.fit interface, this should be done automatically.
• If you use the model as a callable it should accept a training parameter that you can set as training=True to "activate" training mode on a per-call basis. I.e. noisy_image = model(image, training=True).
• You can use tf.keras.backend.set_learning_phase(1) to "globally" activate training mode (call it again later with argument 0 to deactivate).

Answer 2

These data augmentation layers are only active at train time, so will have no effect when simply calling the model on the image. You can see that it's working with:

import tensorflow as tf
import numpy as np
import cv2

stddev = 0.1
image = cv2.imread(<img_path>)
image = (image.astype('float32') - 127.5) / 127.5

input_layer = tf.keras.layers.Input(shape=(128,128,3))
gaus = tf.keras.layers.GaussianNoise(stddev)(input_layer, training=True)
model = tf.keras.models.Model(inputs=input_layer, outputs=gaus)

noisy_image = model(image)

print(f'Pixel value at 0,0: {image[0,0]}')
print(f'Pixel value at 0,0: {noisy_image.numpy()[0,0]}')
# Output
# Pixel value at 0,0: [ 0.09803922 -0.30980393 -0.56078434]
# Pixel value at 0,0: [ 0.09803922 -0.30980393 -0.56078434]

But make sure you remove training=True at evaluation time so it doesn't negatively affect performance.

Answer 3

Try

import matplotlib.pyplot as plt

input_layer = tf.keras.layers.Input(shape=(128,128,3))
gaus        = tf.keras.layers.GaussianNoise(stddev,name='output')(input_layer)
model       = tf.keras.models.Model(inputs=input_layer, outputs=gaus)

noisy_image = get_layer_outputs(model,'output',image,train_mode=True)
plt.imshow(image[:,:,0],cmap='bwr')
plt.show()
plt.imshow(np.asarray(noisy_image)[0,:,:,0],cmap='bwr')
plt.show()

def get_layer_outputs(model,layer_name,input_data,train_mode=False):
    outs_tensor   = [layer.output for layer in model.layers if layer_name == layer.name]
    outs_function = K.function([model.input, K.learning_phase()], outs_tensor)
    return [outs_function([input_data,int(train_mode)])][0]

GaussianNoise only works with learning_phase=1 - i.e. in 'train mode'; unsure if feeding data directly via model(...) accomplishes this. W/ above, using image=np.random.randn(128,128,3) and stddev=1, I get