Tensorflow Graphs : Are Tensorflow graphs DAG? What happens in assignAdd operations in tensor Variables

Question

How is this graph acyclic? assign add op adds x to itself.

import tensorflow as tf
sess = tf.Session()
x = tf.Variable(1300,name="x")
y = tf.Variable(200, name="y")
z = tf.add(x, y,name="z")
b = x.assign_add(z)
init = tf.initialize_all_variables()
writer = tf.train.SummaryWriter("/tmp/logdir", sess.graph)
sess.run(init)
print(sess.run(b))

Clearly there is a bi-directional edge between AssignAdd and X.

Why is X depicted twice as a variable?

Answer 1

As Olivier points out, the graph for your program is a DAG. The graph visualizer takes some liberties when rendering the graph to make it easier to understand. In particular, there are no "bidirectional" edges in the runtime itself, but instead TensorFlow includes "reference edges" for variables, which are like passing a mutable value (like a pointer or a mutable reference) into a C/C++ function, as they allow the recipient to modify the same underlying storage used for the variable.

Note that it is legal for TensorFlow graphs to contain one or more cycles, or even nested cycles. The tf.while_loop() function provides a means of creating structured cycles to represent iterative computations, for which TensorFlow can compute gradients. However, for your use with a simple variable, you do not need a cycle.