Tensorflow: how to use pretrained weights in new graph?

Question

I'm trying to build an object detector with CNN using tensorflow with python framework. I would like to train my model to do just object recognition (classification) at first and then using several convolutional layers of the pretarined model train it to predict bounding boxes. I will need to replace fully connected layers and probably some last convolutional layers. So, for this reason, I would like to know if it is possible to import only weights from tensorflow graph that was used to train object classifier to a newly defined graph that I will train to do object detection. So basically I would like to do something like this:

# here I initialize the new graph
conv_1=tf.nn.conv2d(in, weights_from_old_graph)
conv_2=tf.nn.conv2d(conv_1, weights_from_old_graph)
...
conv_n=tf.nn.nnconv2d(conv_n-1,randomly_initialized_weights)
fc_1=tf.matmul(conv_n, randomly_initalized_weights)

Answer 1

Use saver with no arguments to save the entire model.

tf.reset_default_graph()
v1 = tf.get_variable("v1", [3], initializer = tf.initializers.random_normal)
v2 = tf.get_variable("v2", [5], initializer = tf.initializers.random_normal)
saver = tf.train.Saver()

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    saver.save(sess, save_path='./test-case.ckpt')

    print(v1.eval())
    print(v2.eval())
saver = None

v1 = [ 2.1882825   1.159807   -0.26564872]
v2 = [0.11437789 0.5742971 ]

Then in the model you want to restore to certain values, pass a list of variable names you want to restore or a dictionary of {"variable name": variable} to the Saver.

tf.reset_default_graph()
b1 = tf.get_variable("b1", [3], initializer= tf.initializers.random_normal)
b2 = tf.get_variable("b2", [3], initializer= tf.initializers.random_normal)
saver = tf.train.Saver(var_list={'v1': b1})

with tf.Session() as sess:
  saver.restore(sess, "./test-case.ckpt")
  print(b1.eval())
  print(b2.eval())

INFO:tensorflow:Restoring parameters from ./test-case.ckpt
b1 = [ 2.1882825   1.159807   -0.26564872]
b2 = FailedPreconditionError: Attempting to use uninitialized value b2

Answer 2

Although I agree with Aechlys to restore variables. The problem is harder when we want to fix these variables. For example, we trained these variables and we want to use them in another model, but this time without training them (training new variables like in transfer-learning). You can see the answer I posted here.

Quick example:

 with tf.session() as sess:
    new_saver = tf.train.import_meta_graph(pathToMeta)
    new_saver.restore(sess, pathToNonMeta) 

    weight1 = sess.run(sess.graph.get_tensor_by_name("w1:0")) 


 tf.reset_default_graph() #this will eliminate the variables we restored


 with tf.session() as sess:
    weights = 
       {
       '1': tf.Variable(weight1 , name='w1-bis', trainable=False)
       }
...

We are now sure the restored variables are not a part of the graph.