Is Session.run(fetches) guaranteed to execute its "fetches" arguments in-order?

Question

Is Session.run(fetches, feed_dict) guaranteed to execute its fetches arguments in-order?

The documentation doesn't seem to mention it.

For example, if you run

sess.run([accuracy, train_op], feed_dict=feed_dict)

the order of execution matters: train_op will update parameters affecting accuracy.

Answer 1

No. By default, Tensorflow is free to evaluate operators in any order. Because of concurrency, that order may even change between runs. This is usually a good thing because it means that Tensorflow may make optimal use of the available hardware. It can be problematic if your code mutates state such as Variables.

However, if for some reason you do wish to control the order of evaluation, in general you can use control dependencies to enforce an order between operators. Control dependencies are documented here:

https://www.tensorflow.org/api_docs/python/tf/Graph#control_dependencies

Hope that helps!