Running Adam Optimizer

Question

I am attempting to run an AdamOptimizer for one step of training, unsuccesfully.

optimizer = tf.train.AdamOptimizer(learning_rate)
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    sess.run(optimizer.minimize(cost), feed_dict={X:X_data, Y: Y_data})

The console is spitting an ugly looking error:

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value beta1_power
 [[Node: beta1_power/read = Identity[T=DT_FLOAT, _class=["loc:@W1"], _device="/job:localhost/replica:0/task:0/cpu:0"](beta1_power)]]

In the code, cost is a well defined function implementing a conv NN plus a logistic loss function, using two parameters X, Y (the entry of the NN and the training labels respectively)

Any ideas on what could possibly be wrong?

Answer 1

optimizer.minimize(cost) is creating new values & variables in your graph.

When you call sess.run(init) the variables that the .minimize method creates are not yet defined: from this your error.

You just have to declare your minimization operation before invoking tf.global_variables_initializer():

optimizer = tf.train.AdamOptimizer(learning_rate)
minimize = optimizer.minimize(cost)
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    sess.run(minimize, feed_dict={X:X_data, Y: Y_data})