Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the best way to initialize the Variable in TensorFlow?

In TensorFlow, I can initialize variables in two the ways:

  1. Call global_variable_intializer() before declaration of variable:

    import tensorflow as tf
    
    # Initialize the global variable and session
    init = tf.global_variables_initializer()
    sess = tf.Session()
    sess.run(init)
    
    W = tf.Variable([.3], tf.float32)
    x = tf.Variable([-.3], tf.float32)
    b = tf.Variable([-.3], tf.float32)
    linear_model = W * x + b
    
  2. Call global_variable_intializer() after declaration of variable:

    import tensorflow as tf
    
    W = tf.Variable([.3], tf.float32)
    x = tf.Variable([-.3], tf.float32)
    b = tf.Variable([-.3], tf.float32)
    linear_model = W * x + b 
    
    # Initialize the global variable and session
    init = tf.global_variables_initializer()
    sess = tf.Session()
    sess.run(init)
    

What is the difference between both? Which is the best way to initialize variables?

Edit

This is the actual program I am running :

import tensorflow as tf

# Initialize the global variable and session
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)

W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)

x = tf.placeholder(tf.float32)
y = tf.placeholder(tf.float32)



linear_model = W * x + b

square_delta = tf.square(linear_model - y)

loss = tf.reduce_sum(square_delta)

fixW = tf.assign(W, [-1.])
fixb = tf.assign(b, [1.])

sess.run([fixW, fixb])

print(sess.run(loss, {x:[1,2,3,4], y:[0,-1,-2,-3]}))
like image 386
Pankit Gami Avatar asked May 31 '17 12:05

Pankit Gami


1 Answers

In the case 1, the variables are not initialized, and if you try

sess.run(linear_model)

it should give you some kind of error (FailedPreconditionError on my compiler).

Case 2 is the working one.

The command

tf.global_variables_initializer()

should be called after all variables are created, otherwise the same error will be raised.

As far as I understand, each time you call tf.Variable, the nodes related to a variable are added to the graph. These are the following:

Variable/initial_value
Variable
Variable/Assign
Variable/read

(you obtain the nodes constructed so far with the command

for n in tf.get_default_graph().as_graph_def().node:
    print n.name

)

The variable itself, has no value until you run within a Session the Variable/Assign node.

The command

init = tf.global_variables_initializer() 

creates a single node containing all the assign nodes of all the variables constructed so far, and associate it to the python variable 'init' so that when it is executed the line

sess.run(init)

all variables acquire the initial value.

like image 114
Pietro Tortella Avatar answered Sep 30 '22 04:09

Pietro Tortella