How can I change a network after initialising using functional API?

Question

I have a rather "simple" question. When I create a network using the functional API:

layer2 = Dense(8, name="layer2")(layer1)

and then initialise it with

model = Model(input=..., output=...)

what can I do if I want to change layers afterwards? If I .pop() and then .append() a new layer, nothing changes - the output stays the same. I think this is because the output is still defined beforehand.

The exact problem I have is this: I load a pre-trained AlexNet with its weights but then I would like to retrain the last Dense layer for a classification task of 8 classes instead of 1000. For this I wanted to drop the last layers and re-add them.

I found a workaround (Changing pretrained AlexNet classification in Keras) but I think there should be an easier way. Additionally, I dont think my workaround will work with a GoogLeNet so I would really love to know (or a hint) how to handle this situation.

Answer 1

The Model object does not hold the weights, the layers do. You can load the weights for your model using model.load_weights() and then create a new layer based on the layers you have without losing the initialization of the layers.

For example:

model.load_weights(f)
newClassificationLayer = Dense(8, activation='softmax')(lastCnnLayer)
model = Model(input=oldInput, output=newClassificationLayer)

To make sure all other layers are frozen and do not get trained except for your new layer you can set trainable=False for each layer you want to freeze. E.g.:

lastCnnLayer.trainable = False