How can I reuse a Dense layer?

Question

I have a network in Tensorflow, and I want to define a function that passes it's input through a tf.layers.dense layer (obviously, the same one). I see the reuse argument, but in order to use it properly it seems I need to keep a global variable just to remember if my function was called already. Is there a cleaner way?

Answer 1

I find tf.layers.Dense cleaner than the above answers. All you need is a Dense object defined beforehand. Then you can reuse it any number of times.

import tensorflow as tf

# Define Dense object which is reusable
my_dense = tf.layers.Dense(3, name="optional_name")

# Define some inputs
x1 = tf.constant([[1,2,3], [4,5,6]], dtype=tf.float32)
x2 = tf.constant([[4,5,6], [7,8,9]], dtype=tf.float32)

# Use the Dense layer
y1 = my_dense(x1)
y2 = my_dense(x2)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    y1 = sess.run(y1)
    y2 = sess.run(y2)
    print(y1)
    print(y2)

In fact tf.layers.dense function internally constructs a Dense object and pass your input to that object. For more details, check the code.

Answer 2

As far as I know, there's no cleaner way. The best we can do is wrap tf.layers.dense into our abstraction and use it as an object, hiding variable scope's backbone:

def my_dense(*args, **kwargs):
  scope = tf.variable_scope(None, default_name='dense').__enter__()
  def f(input):
    r = tf.layers.dense(input, *args, name=scope, **kwargs)
    scope.reuse_variables()
    return r
  return f

a = [[1,2,3], [4,5,6]]
a = tf.constant(a, dtype=tf.float32)
layer = my_dense(3)
a = layer(a)
a = layer(a)

print(*[[int(a) for a in v.get_shape()] for v in tf.trainable_variables()])
# Prints: "[3, 3] [3]" (one pair of (weights and biases))