How to get weight matrix of one layer at every epoch in LSTM model based on Keras?

Question

I have a simple LSTM model based on Keras.

X_train, X_test, Y_train, Y_test = train_test_split(input, labels, test_size=0.2, random_state=i*10)

X_train = X_train.reshape(80,112,12)
X_test = X_test.reshape(20,112,12)

y_train = np.zeros((80,112),dtype='int')
y_test = np.zeros((20,112),dtype='int')

y_train = np.repeat(Y_train,112, axis=1)
y_test = np.repeat(Y_test,112, axis=1)
np.random.seed(1)

# create the model
model = Sequential()
batch_size = 20

model.add(BatchNormalization(input_shape=(112,12), mode = 0, axis = 2))#4
model.add(LSTM(100, return_sequences=False, input_shape=(112,12))) #7 

model.add(Dense(112, activation='hard_sigmoid'))#9
model.compile(loss='binary_crossentropy', optimizer='RMSprop', metrics=['binary_accuracy'])#9

model.fit(X_train, y_train, nb_epoch=30)#9

# Final evaluation of the model
scores = model.evaluate(X_test, y_test, batch_size = batch_size, verbose=0)

I know how to get the weight list by model.get_weights(), but that's the value after the model is fully trained. I want to get the weight matrix(for example, the last layer in my LSTM) at every epoch rather than only the final value of it. In other words, I have 30 epochs and I need to get 30 weight matrix values.

Really thank you, I didn't find the solution on the wiki of keras.

Answer 1

You can write a custom callback for it:

from keras.callbacks import Callback

class CollectWeightCallback(Callback):
    def __init__(self, layer_index):
        super(CollectWeightCallback, self).__init__()
        self.layer_index = layer_index
        self.weights = []

    def on_epoch_end(self, epoch, logs=None):
        layer = self.model.layers[self.layer_index]
        self.weights.append(layer.get_weights())

The attribute self.model of a callback is a reference to the model being trained. It is set via Callback.set_model() when training starts.

To get the weights of the last layer at each epoch, use it with:

cbk = CollectWeightCallback(layer_index=-1)
model.fit(X_train, y_train, nb_epoch=30, callbacks=[cbk])

The weight matrices will then be collected into cbk.weights.