Restoring specific TensorFlow variables to a specific layer (Restore by name)

Question

Suppose I trained a TensorFlow model and saved it, now have a different model, and I want to use some of the weights in the saved model for some of the layers in my model (they have the same shape).

Now, I was able to find how to save specific variables from a model (with specific names) but I wasn't able to find any example for restoring those variables by name.

For example suppose that in my saved model I saved a tensor of weights (with some shape) called "v1". Now in my new model I have a weights tensor called "v2" (which have the same shape of the "v1" tensor). Now I want to load the saved variables of "v1" to my "v2" weights tensor, or even better load this "v1" weights to multiple tensor in my new graph.

Is that even possible? If so, how do I do it?

Answer 1

I found a workaround to solve this issue.

What you can do is to save the variable values directly to you disk, either as a value or as a dictionary with keys as tensor names and values. For example:

vars_dict = {}
for tensor in (list_of_tensors_you_want_to_save):
     vars_dict[tensor.name] = sess.run(tensor)

Then you can load any variable from this dictionary to any other variable you want.

In my example, suppose the original tensor is called "v1" and the two tensor I want to load are "v2" and "v3", the following can be done:

tensor_to_load_1 = tf.get_default_graph().get_tensor_by_name("v2")
tensor_to_load_2 = tf.get_default_graph().get_tensor_by_name("v3")

assign_op_1 = tf.assign(tensor_to_load_1, vars_dict["v1"])
assign_op_1 = tf.assign(tensor_to_load_2, vars_dict["v1"])

sess.run([assign_op_1, assign_op_2])

This, of course, is only limited by the fact that "v1", "v2" and "v3" must have the same shape.

Using this sample code you can save any variables and load them to any other variables you want, without the need for the original graph to match your current one.