Tensorflow 1.8.0: Wide and Deep Model results are not stable. Random seed is not working

Question

I've trained a Wide and Deep Model using Tensorflow 1.8.0. My test and training dataset are separate files which were split previously. I'm using tf.set_random_seed(1234) before tf.contrib.learn.DNNLinearCombinedClassifier as follows -

tf.set_random_seed(1234)

import tempfile

model_dir = tempfile.mkdtemp()
m = tf.contrib.learn.DNNLinearCombinedClassifier(model_dir=model_dir,
                                                 linear_feature_columns=wide_columns,
                                                 dnn_feature_columns=deep_columns,
                                                 dnn_hidden_units=[100, 50])

It shows the following logs-

INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_task_type': None, '_task_id': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f394b585c18>, '_master': '', '_num_ps_replicas': 0, '_num_worker_replicas': 0, '_environment': 'local', '_is_chief': True, '_evaluation_master': '', '_train_distribute': None, '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1.0
}
, '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_secs': 600, '_log_step_count_steps': 100, '_session_config': None, '_save_checkpoints_steps': None, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_model_dir': '/tmp/tmpxka6vy6t'}

From the log, I can see that the random seed was not applied.

Whenever I run the script I get different accuracy results.

How can I make the result stable? Why random seed is not getting applied?

Answer 1

After so much struggle, finally, I've found the solution. The tf_random_seed need to be set inside the DNNLinearCombinedClassifier as an argument of config. Including the line config=tf.contrib.learn.RunConfig(tf_random_seed=123) solves the problem. It sets the random seed and makes the result reproducible.

Here is how the code should be look like -

# Combining Wide and Deep Models into One
model_dir = tempfile.mkdtemp()
m = tf.contrib.learn.DNNLinearCombinedClassifier(model_dir=model_dir,
                                                 linear_feature_columns=wide_columns,
                                                 dnn_feature_columns=deep_columns,
                                                 dnn_hidden_units=[100, 50],
                                                 config=tf.contrib.learn.RunConfig(tf_random_seed=123))