How to restore weights with different names but same shapes Tensorflow?

Question

I have multiple architectures in Tensorflow. Some of them share the design of certain parts.

I would like to train one of the networks and use the trained weights of the similar layers in another network.

At this point in time, I am able to save the weights I want and reload them in an architecture with an exactly similar naming convention for the variables.

However, when the weights have different names in the two networks, it is not possible to restore. I have this naming convention for the first network:

• selector_network/c2w/var1

in the second network I have this:

• joint_network/c2w/var1

Apart from that, the variables are similar in terms of shape. Is there a possibility to change the names upon reloading or to tell Tensorflow where to fit those variables?

EDIT: I found this script from @batzner that allows renaming the variables of a Tensorflow checkpoint : tensorflow_rename_variables.

It is not working. I get the following error:

ValueError: Couldn't find 'checkpoint' file or checkpoints in given directory ./joint_pos_tagger_lemmatizer/fi/

Answer 1

tf.train.Saver has builtin support for that using a dictionary for the var_list argument. This dictionary maps the names of the objects in the checkpoint file to your variables you want to restore.

If you want to restore your "joint network" with a checkpoint of your "selector network", you can do it like this:

# var1 is the variable you want ot restore
saver = tf.train.Saver(var_list={'selector_network/c2w/var1': var1})
saver.restore(...)

If you want to restore more variables, you simply have to extend the dictionary.