Tensorflow variable scope: reuse if variable exists

Question

I want a piece of code that creates a variable within a scope if it doesn't exist, and access the variable if it already exists. I need it to be the same code since it will be called multiple times.

However, Tensorflow needs me to specify whether I want to create or reuse the variable, like this:

with tf.variable_scope("foo"): #create the first time     v = tf.get_variable("v", [1])  with tf.variable_scope("foo", reuse=True): #reuse the second time     v = tf.get_variable("v", [1]) 

How can I get it to figure out whether to create or reuse it automatically? I.e., I want the above two blocks of code to be the same and have the program run.

Answer 1

A ValueError is raised in get_variable() when creating a new variable and shape is not declared, or when violating reuse during variable creation. Therefore, you can try this:

def get_scope_variable(scope_name, var, shape=None):     with tf.variable_scope(scope_name) as scope:         try:             v = tf.get_variable(var, shape)         except ValueError:             scope.reuse_variables()             v = tf.get_variable(var)     return v  v1 = get_scope_variable('foo', 'v', [1]) v2 = get_scope_variable('foo', 'v') assert v1 == v2 

Note that the following also works:

v1 = get_scope_variable('foo', 'v', [1]) v2 = get_scope_variable('foo', 'v', [1]) assert v1 == v2 

---

UPDATE. The new API supports auto-reusing now:

def get_scope_variable(scope, var, shape=None):     with tf.variable_scope(scope, reuse=tf.AUTO_REUSE):         v = tf.get_variable(var, shape)     return v 

Answer 2

Although using "try...except..." clause works, I think a more elegant and maintainable way would be separate the variable initialization process with the "reuse" process.

def initialize_variable(scope_name, var_name, shape):     with tf.variable_scope(scope_name) as scope:         v = tf.get_variable(var_name, shape)         scope.reuse_variable()  def get_scope_variable(scope_name, var_name):     with tf.variable_scope(scope_name, reuse=True):         v = tf.get_variable(var_name)     return v 

Since often we only need to initialize variable ones, but reuse/share it for many times, separating the two processes make the code cleaner. Also this way, we won't need to go through the "try" clause every time to check if the variable has been created already or not.