What's the difference between Variable and ResourceVariable in Tensorflow

Question

In Tensorflow, Variable is a resource, inherited from ResourceBase and managed by ResourceMgr. But why is there another named ResourceVariable? Both of them can be used for optimizers like gradient_descent (see this example). What's the difference? I know the former is well documented and most often used. What's the purpose of the latter?

Answer 1

ResourceVariable is the replacement for Variable, that aims to clean up some of the messier aspects of the semantics of Variable.

ResourceVariable is the default in TF 2.0 and you very likely don't care about the differences between the two unless you are working on details deep inside the Tensorflow implementation. When eager execution is enabled tf.Variable also creates resource variables.

So just use tf.Variable for now, it's almost certainly what you want; if you experience issues which look like race conditions or bugs from inconsistent values of variables in code you can try enabling resource variables (by either passing use_resource=True to your variable-creating code or calling tf.enable_resource_variables() in TF 1.x).

Answer 2

From the comments:

Unlike tf.Variable, a tf.ResourceVariable has well-defined semantics. Each usage of a ResourceVariable in a TensorFlow graph adds a read_value operation to the graph. The Tensors returned by a read_value operation are guaranteed to see all modifications to the value of the variable which happen in any operation on which the read_value depends on (either directly, indirectly, or via a control dependency) and guaranteed to not see any modification to the value of the variable on which the read_value operation does not depend on. For example, if there is more than one assignment to a ResourceVariable in a single session.run call there is a well-defined value for each operation which uses the variable's value if the assignments and the read are connected by edges in the graph. Consider the following example, in which two writes can cause tf.Variable and tf.ResourceVariable to behave differently:

a = tf.ResourceVariable(1.0)
a.initializer.run()
assign = a.assign(2.0)
with tf.control_dependencies([assign]):
    b = a.read_value()
with tf.control_dependencies([b]):
    other_assign = a.assign(3.0)
with tf.control_dependencies([other_assign]):
    # Will print 2.0 because the value was read before other_assign ran. If
    # `a` was a tf.Variable instead, 2.0 or 3.0 could be printed.
    tf.Print(b, [b]).eval()

To enforce these consistency properties tf.ResourceVariable might make more copies than an equivalent tf.Variable under the hood, so tf.Variable is still not deprecated.