How to add and remove new layers in keras after loading weights?

Question

I am trying to do a transfer learning; for that purpose I want to remove the last two layers of the neural network and add another two layers. This is an example code which also output the same error.

from keras.models import Sequential from keras.layers import Input,Flatten from keras.layers.convolutional import Convolution2D, MaxPooling2D from keras.layers.core import Dropout, Activation from keras.layers.pooling import GlobalAveragePooling2D from keras.models import Model  in_img = Input(shape=(3, 32, 32)) x = Convolution2D(12, 3, 3, subsample=(2, 2), border_mode='valid', name='conv1')(in_img) x = Activation('relu', name='relu_conv1')(x) x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool1')(x) x = Convolution2D(3, 1, 1, border_mode='valid', name='conv2')(x) x = Activation('relu', name='relu_conv2')(x) x = GlobalAveragePooling2D()(x) o = Activation('softmax', name='loss')(x) model = Model(input=in_img, output=[o]) model.compile(loss="categorical_crossentropy", optimizer="adam") #model.load_weights('model_weights.h5', by_name=True) model.summary()  model.layers.pop() model.layers.pop() model.summary() model.add(MaxPooling2D()) model.add(Activation('sigmoid', name='loss')) 

I removed the layer using pop() but when I tried to add its outputting this error

AttributeError: 'Model' object has no attribute 'add'

I know the most probable reason for the error is improper use of model.add(). what other syntax should I use?

EDIT:

I tried to remove/add layers in keras but its not allowing it to be added after loading external weights.

from keras.models import Sequential from keras.layers import Input,Flatten from keras.layers.convolutional import Convolution2D, MaxPooling2D from keras.layers.core import Dropout, Activation from keras.layers.pooling import GlobalAveragePooling2D from keras.models import Model in_img = Input(shape=(3, 32, 32))  def gen_model():     in_img = Input(shape=(3, 32, 32))     x = Convolution2D(12, 3, 3, subsample=(2, 2), border_mode='valid', name='conv1')(in_img)     x = Activation('relu', name='relu_conv1')(x)     x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool1')(x)     x = Convolution2D(3, 1, 1, border_mode='valid', name='conv2')(x)     x = Activation('relu', name='relu_conv2')(x)     x = GlobalAveragePooling2D()(x)     o = Activation('softmax', name='loss')(x)     model = Model(input=in_img, output=[o])     return model  #parent model model=gen_model() model.compile(loss="categorical_crossentropy", optimizer="adam") model.summary()  #saving model weights model.save('model_weights.h5')  #loading weights to second model model2=gen_model() model2.compile(loss="categorical_crossentropy", optimizer="adam") model2.load_weights('model_weights.h5', by_name=True)  model2.layers.pop() model2.layers.pop() model2.summary()  #editing layers in the second model and saving as third model x = MaxPooling2D()(model2.layers[-1].output) o = Activation('sigmoid', name='loss')(x) model3 = Model(input=in_img, output=[o]) 

its showing this error

RuntimeError: Graph disconnected: cannot obtain value for tensor input_4 at layer "input_4". The following previous layers were accessed without issue: [] 

Answer 1

You can take the output of the last model and create a new model. The lower layers remains the same.

model.summary() model.layers.pop() model.layers.pop() model.summary()  x = MaxPooling2D()(model.layers[-1].output) o = Activation('sigmoid', name='loss')(x)  model2 = Model(input=in_img, output=[o]) model2.summary() 

Check How to use models from keras.applications for transfer learnig?

Update on Edit:

The new error is because you are trying to create the new model on global in_img which is actually not used in the previous model creation.. there you are actually defining a local in_img. So the global in_img is obviously not connected to the upper layers in the symbolic graph. And it has nothing to do with loading weights.

To better resolve this problem you should instead use model.input to reference to the input.

model3 = Model(input=model2.input, output=[o])

Answer 2

Another way to do it

from keras.models import Model  layer_name = 'relu_conv2' model2= Model(inputs=model1.input, outputs=model1.get_layer(layer_name).output)