Is tensorflow lazy?

Question

Let's say you have some piece of code like this

import tensorflow as tf
...
f = h*y + z*t  #Just some expression involving other tensors.
e = ... # some expression that does not involve f. 
result = tf.select(b, e, f)

sess.run(result)

b is a boolean tensor of the same shape as e and f. If all the elements of b evaluate to true, we don't need f and the result will just be (or be equal to) e.

The question: when the session is run with result, and the elements of e are all true, is f evaluated?

Answer 1

TL;DR: TensorFlow is strict, so both e and f will be evaluated before the tf.select() node executes.

This has caused some confusion. TensorFlow first prunes the dataflow graph based on which operations are statically required to produce the values that are fetched (i.e. the arguments to sess.run()). Once the graph has been pruned, however, the runtime uses strict execution, whereby all of the inputs to an operation (such as tf.select()) must have been computed before that operation can execute.

There is experimental support for conditional execution in the tf.control_flow_ops module, using the tf.control_flow_ops.cond() function, but this is sparsely documented at present.