How does TensorFlow name tensors?

Question

I wonder if this is the correct understanding:

All tensors are derived from some operation, and operations are either given a name in the constructor, or given the default name for a particular kind of operation. If the name is not unique, TensorFlow automatically handles this by appending "_1", "_2", etc. An operation with n tensor outputs name these tensors "op_name:0", "op_name:1", ..., "op_name:n-1".

One problem seems to arise: if x is a tf.Variable, then x.name gives "variable_name:0". This is confusing: to what does "variable_name" refer?

Answer 1

Your observations on Tensor naming are absolutely correct: the name of a Tensor is the concatenation of

1. the name of the operation that produced it,
2. a colon (:), and
3. the index of that tensor in the outputs of the operation that produced it.

Therefore the tensor named "foo:2" is the output of the op named "foo" at position 2 (with indices starting from zero).

The naming of tf.Variable objects is slightly strange. Every tf.Variable contains a mutable tensor object that holds the state of the variable (and a few other tensors). A "Variable" op (which has the name "variable_name" in your example) "produces" this mutable tensor each time it is run as its 0th output, so the name of the mutable tensor is "variable_name:0".

Since a tf.Variable is mostly indistinguishable from a tf.Tensor—in that it can be used in the same places—we took the decision to make variable names resemble tensor names, so the Variable.name property returns the name of the mutable tensor. (This contrasts with tf.QueueBase and tf.ReaderBase objects, which are not usable directly as tensors (instead you have to call methods on them to create ops that operate on their state), so these do not have a tensor-like name.)