Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to set parameters in keras to be non-trainable?

I am new to Keras and I am building a model. I want to freeze the weights of the last few layers of the model while training the previous layers. I tried to set the trainable property of the lateral model to be False, but it dosen't seem to work. Here is the code and the model summary:

opt = optimizers.Adam(1e-3)
domain_layers = self._build_domain_regressor()
domain_layers.trainble = False
feature_extrator = self._build_common()
img_inputs = Input(shape=(160, 160, 3))
conv_out = feature_extrator(img_inputs)
domain_label = domain_layers(conv_out)
self.domain_regressor = Model(img_inputs, domain_label)
self.domain_regressor.compile(optimizer = opt, loss='binary_crossentropy', metrics=['accuracy'])
self.domain_regressor.summary()

The model summary: model summary.

As you can see, model_1 is trainable. But according to the code, it is set to be non-trainable.

like image 818
Swi Jason Avatar asked Nov 27 '18 15:11

Swi Jason


People also ask

How do you make a keras layer not trainable?

This leads us to how a typical transfer learning workflow can be implemented in Keras: Instantiate a base model and load pre-trained weights into it. Freeze all layers in the base model by setting trainable = False .

What are non trainable parameters in keras?

Non trainable parameters are those which value is not optimized during the training as per their gradient.

Why do I have non trainable params?

These parameters are "non-trainable" because you can't optimize its value with your training data. Training algorithms (like back-propagation) will optimize and update the weights of your network, which are the actual trainable parameters here (usually several thousands, depending on your connections).


1 Answers

You can simple assign a boolean value to the layer property trainable.

model.layers[n].trainable = False

You can visualize which layer is trainable:

for l in model.layers:
    print(l.name, l.trainable)

You can pass it by the model definition too:

frozen_layer = Dense(32, trainable=False)

From Keras documentation:

To "freeze" a layer means to exclude it from training, i.e. its weights will never be updated. This is useful in the context of fine-tuning a model, or using fixed embeddings for a text input.
You can pass a trainable argument (boolean) to a layer constructor to set a layer to be non-trainable. Additionally, you can set the trainable property of a layer to True or False after instantiation. For this to take effect, you will need to call compile() on your model after modifying the trainable property.

like image 147
Geeocode Avatar answered Oct 12 '22 02:10

Geeocode