Saving layer weights at each epoch during training into a numpy type/array? Converting TensorFlow Variable to numpy array?

Question

I am working with keras for the first time and am attempting to write a custom keras.callbacks.Callback which saves the weights of each model layer during fit. I am having trouble converting the type of keras.models.layers.weights to a numpy array (or anything from which I can extract the primitive type value).

From what I can tell keras.models.layers.weights (for my model) is a list of tensorflow.python.ops.variables.Variable that hold either a (1, 1) matrix or (1,) matrix. I just cannot seem to get this variable type as a float (the dtype of the matrices).

Here is a SSCCE of my issue.

import keras
import numpy as np

xONE = np.mat([[0], [1]])
yNOT = np.mat([[1], [0]])

# Simple Keras callback which saves not only loss and acc
# but also the layer weights at each epoch end
class History_LAW(keras.callbacks.Callback):
    def on_train_begin(self, logs={}):
        self.epoch = []
        self.weights = []
        self.history = {}
        self.weights.append(self.model.weights)

    def on_epoch_end(self, epoch, logs={}):
        logs = logs or {}
        self.epoch.append(epoch)
        self.weights.append(self.model.weights)
        for k, v in logs.items():
            self.history.setdefault(k, []).append(v)

# Used to programmatically generaly a simple model
# given the data that it is meant to fit to
def generateModel(xShape, yShape):
    model = keras.models.Sequential()
    model.add(
        keras.layers.Dense(
            yShape[1], input_shape = (xShape[1],)))
    model.add(keras.layers.Activation('hard_sigmoid'))
    model.summary()
    return model

# Generate a model based on the simply NOT relationship
model = generateModel(xONE.shape, yNOT.shape)
model.compile(
    optimizer = 'adam', 
    loss = 'binary_crossentropy', 
    metrics = ['accuracy'])

# Create the custom callback and pass it to fit
model_Hist = History_LAW()
model.fit(xONE, yNOT, 
    epochs = 4, batch_size = 2, verbose = 1, callbacks = [model_Hist])

# Display the raw weight structure and its type
print('========== Raw w output')
for weight in model_Hist.weights:
    print([w for w in weight])

print('========== Type of w')
for weight in model_Hist.weights:
    print([type(w) for w in weight])

I did find this SO answer which proposes to use eval() (I think a TensorFlow function) to "evalute" the tensors (not exactly sure what that means yet but I think thats the proper terminology) into their underlying type -- Which I assume is a numpy matrix or array. Though follow this seems to lead down another rabbit hole of errors (passing session = tf.InteractiveSession) to eval, resulting in "the tensor's graph is different from the session's graph.").

I have found various callbacks which can record layer weights during fitting though from what I can tell none of them return them in the desired format.

How can I convert these weights to some numpy object? Is there an easier way other than what I propose in the SSCCE to accomplish what I want?

---

For reference the output of the SSCCE above is:

========== Raw w output
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
========== Type of w
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]

Answer 1

I have found that I need to use the method keras.model.layer.get_weights() to get the weights as numpy arrays.

For instance my callback would change to something like

def on_epoch_end(self, epoch, logs={}):
    logs = logs or {}
    self.epoch.append(epoch)
    for k, v in logs.items():
        self.history.setdefault(k, []).append(v)

    modelWeights = []
    for layer in model.layers:
        layerWeights = []
        for weight in layer.get_weights():
            layerWeights.append(weight)
        modelWeights.append(layerWeights)
    self.weights.append(modelWeights)