How can I get a tensor output by a tensorflow.layer

Question

I created a CNN model using higher level tensorflow layers, like

conv1 = tf.layers.conv2d(...)
maxpooling1 = tf.layers.max_pooling2d(...)
conv2 = tf.layers.conv2d(...)
maxpooling2 = tf.layers.max_pooling2d(...)
flatten = tf.layers.flatten(...)
logits = tf.layers.dense(...)
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(...))
optimizer = tf.train.AdadeltaOptimizer(init_lr).minimize(loss)
acc = tf.reduce_mean(...)

The model is well trained and saved, everything is good so far. Next, I want to load this saved model, make a change to the learning rate, and continue to train (I know tensorflow provides exponential_decay() function to allow a decay learning rate, here i just want to be in full control of learning rate, and change it manually). To do this, my idea is like:

saver = tf.train.import_meta_grah(...)
saver.restore(sess, tf.train.latest_chechpoint(...))
graph = tf.get_default_graph()

inputImg_ = graph.get_tensor_by_name(...)  # this is place_holder in model
labels_ = graph.get_tensor_by_name(...)  # place_holder in model
logits = graphget_tensor_by_name(...) # output of dense layer
loss = grah.get_tensor_by_name(...) # loss
optimizer = tf.train.AdadeltaOptimizer(new_lr).minimize(loss) # I give it a new learning rate
acc = tf.reduce_mean(...)

Now I got a problem. the code above can successfully obtain inputmg_, labels_, because I named them when I defined them. But I cannot obtain logits because logits = tf.layers.dense(name='logits') the name is actually given to the dense layer instead of the output tensor logits. That means, I cannot obtain the tensor conv1, conv2 either. It seems tensorflow cannot name a tensor output by a layer. In this case, is there a way to obtain these tensors, like logits, conv1, maxpooling1? I've searched for the answer for a while but failed.

Answer 1

I was having the same problem and solved it using tf.identity.

Since the dense layer has bias and weights parameters, when you name it, you are naming the layer, not the output tensor.

The tf.identity returns a tensor with the same shape and contents as input.

So just leave the dense layer unamed and use it as input to the tf.identity

self.output = tf.layers.dense(hidden_layer3, 2)
self.output = tf.identity(self.output, name='output')

Now you can load the output

output = graph.get_tensor_by_name('output:0')