What is a local variable in tensorflow?

Question

Tensorflow has this API defined:

tf.local_variables()

Returns all variables created with collection=[LOCAL_VARIABLES].

Returns:

A list of local Variable objects.

What exactly is a local variable in TensorFlow? Can someone give me an example?

Answer 1

Short answer: a local variable in TF is any variable which was created with collections=[tf.GraphKeys.LOCAL_VARIABLES]. For example:

e = tf.Variable(6, name='var_e', collections=[tf.GraphKeys.LOCAL_VARIABLES])

LOCAL_VARIABLES: the subset of Variable objects that are local to each machine. Usually used for temporarily variables, like counters. Note: use tf.contrib.framework.local_variable to add to this collection.

They are usually not saved/restored to checkpoint and used for temporary or intermediate values.

---

Long answer: this was a source of confusion for me as well. In the beginning I thought that local variables mean the same thing as local variable in almost any programming language, but it is not the same thing:

import tensorflow as tf

def some_func():
    z = tf.Variable(1, name='var_z')

a = tf.Variable(1, name='var_a')
b = tf.get_variable('var_b', 2)
with tf.name_scope('aaa'):
    c = tf.Variable(3, name='var_c')

with tf.variable_scope('bbb'):
    d = tf.Variable(3, name='var_d')

some_func()
some_func()

print [str(i.name) for i in tf.global_variables()]
print [str(i.name) for i in tf.local_variables()]

No matter what I tried, I always recieved only global variables:

['var_a:0', 'var_b:0', 'aaa/var_c:0', 'bbb/var_d:0', 'var_z:0', 'var_z_1:0']
[]

The documentation for tf.local_variables have not provided a lot of details:

Local variables - per process variables, usually not saved/restored to checkpoint and used for temporary or intermediate values. For example, they can be used as counters for metrics computation or number of epochs this machine has read data. The local_variable() automatically adds new variable to GraphKeys.LOCAL_VARIABLES. This convenience function returns the contents of that collection.

---

But reading docs for the init method in tf.Variable class, I found that while creating a variable, you can provide what kind of a variable do you want it to be by assigning a list of collections.

The list of possible collection elements is here. So to create a local variable you need to do something like this. You will see it in the list of local_variables:

e = tf.Variable(6, name='var_e', collections=[tf.GraphKeys.LOCAL_VARIABLES])
print [str(i.name) for i in tf.local_variables()]

Answer 2

It's the same as regular variable, but it's in a different collection than default (GraphKeys.VARIABLES). That collection is used by saver to initialize the default list of variables to save, so having a local designation has an effect of not saving that variable by default.

I'm seeing only one place that uses it in the codebase, which is the limit_epochs

  with ops.name_scope(name, "limit_epochs", [tensor]) as name:
    zero64 = constant_op.constant(0, dtype=dtypes.int64)
    epochs = variables.Variable(
        zero64, name="epochs", trainable=False,
        collections=[ops.GraphKeys.LOCAL_VARIABLES])