How to turn off dropout for testing in Tensorflow?

Question

I am fairly new to Tensorflow and ML in general, so I hereby apologize for a (likely) trivial question.

I use the dropout technique to improve learning rates of my network, and it seems to work just fine. Then, I would like to test the network on some data to see if it works like this:

   def Ask(self, image):         return self.session.run(self.model, feed_dict = {self.inputPh: image}) 

Obviously, it yields different results each time as the dropout is still in place. One solution I can think of is to create two separate models - one for a training and the other one for an actual later use of the network, however, such a solution seems impractical to me.

What's the common approach to solving this problem?

Answer 1

The easiest way is to change the keep_prob parameter using a placeholder_with_default:

prob = tf.placeholder_with_default(1.0, shape=()) layer = tf.nn.dropout(layer, prob) 

in this way when you train you can set the parameter like this:

sess.run(train_step, feed_dict={prob: 0.5}) 

and when you evaluate the default value of 1.0 is used.

Answer 2

With the new tf.estimator API you specify a model function, that returns different models, based on whether you are training or testing, but still allows you to reuse your model code. In your model function you would do something similar to:

def model_fn(features, labels, mode):      training = (mode == tf.estimator.ModeKeys.TRAIN)     ...     t = tf.layers.dropout(t, rate=0.25, training=training, name='dropout_1')     ... 

The mode argument is automatically passed depending on whether you call estimator.train(...) or estimator.predict(...).