What is the sequence for tensorflow's session to run a list of tensors?

Question

See the code snippet:

import tensorflow as tf

x = tf.Variable(1)
op = tf.assign(x, x + 1)

with tf.Session() as sess:
  tf.global_variables_initializer().run()
  print(sess.run([x, op]))

There are two possible results:

• x=1 and op=2
• x=2 and op=2

They depend on the order of evaluation, for the first case, x is evaluated before op, and for the second case, x is evaluated after op.

I have run the code many times, but the result is always x=2 and op=2. So I guess that tensorflow can guarantee x is evaluated after op. Is it right? And how does tensorflow guarantee the dependence?

Update

For the case above, the result is determinate. But in the follow case, the result is not determinate.

import tensorflow as tf

x = tf.Variable(1)
op = tf.assign(x, x + 1)
x = x + 0                   # add this line

with tf.Session() as sess:
  tf.global_variables_initializer().run()
  for i in range(5):
    print(sess.run([x, op]))

In the first code, x is Variable and op depends on x, so x is always evaluated after op. But in the second case, x becomes Tensor, and op depend on Variable x(After x = x + 0, x is overrided). So the op doesn't depend on Tensor x.

Answer 1

The order in which tensors are evaluated is undefined. See the API docs (towards, the very bottom, in the "Returns" info on Session.run()). As such, you should not rely on them being executed in a particular order. If you need to guarantee an order you should probably use separate run calls for the different tensors/ops.