Add control dependency after operations are created?

Question

Is it possible to create a control dependency between two ops after they've both been created? I realize that with tf.control_dependencies it's possible to make one op wait on another before executing, but the dependent op has to be created within the tf.control_dependencies context. I'd like to construct the two ops independently first and then add the dependency.

Answer 1

This might be a disappointing answer, but it is not possible to add a control dependency (or any other input) to a TensorFlow Operation after it has been created. Tensors and operations are immutable once they have been created.

One possibility would be to clone the op that should run second, in the appropriate control dependencies context. Let's say that you have two Operation objects, op_first and op_second, and you want op_first to run before op_second:

def first_before_second(op_first, op_second):
    """Sequence `op_first` before `op_second`.

    Given two operations, returns a pair of operations with the same behavior
    except that the first returned operation will execute before the second
    returned operation.
    """
    with tf.control_dependencies([op_first]):
        g = tf.get_default_graph()
        op_second_clone = g.create_op(op_second.type,
                                      [in_t for in_t in op_second.inputs],
                                      [out_t.dtype for out_t in op_second.outputs],
                                      attrs=op_second.node_def.attr,
                                      op_def=op_second.op_def)

    return op_first, op_second_clone

A similar modification could make this deal with Tensor objects as well, but that's left as an exercise for the reader.