Is it possible to modify an existing TensorFlow computation graph?

Question

TensorFlow graph is usually built gradually from inputs to outputs, and then executed. Looking at the Python code, the inputs lists of operations are immutable which suggests that the inputs should not be modified. Does that mean that there is no way to update/modify an existing graph?

Answer 1

The TensorFlow tf.Graph class is an append-only data structure, which means that you can add nodes to the graph after executing part of the graph, but you cannot remove or modify existing nodes. Since TensorFlow executes only the necessary subgraph when you call Session.run(), there is no execution-time cost to having redundant nodes in the graph (although they will continue to consume memory).

To remove all nodes in the graph, you can create a session with a new graph:

with tf.Graph().as_default():  # Create a new graph, and make it the default.   with tf.Session() as sess:  # `sess` will use the new, currently empty, graph.     # Build graph and execute nodes in here. 

Answer 2

Yes, tf.Graph are build in an append-only fashion as @mrry puts it.

But there's workaround:

Conceptually you can modify an existing graph by cloning it and perform the modifications needed along the way. As of r1.1, Tensorflow provides a module named tf.contrib.graph_editor which implements the above idea as a set of convinient functions.