set_model() missing 1 required positional argument: 'model'

Question

I'm have created a Keras Sequential Model and am using Adam optimizer. I wished to get the learning rate after every epoch. This stackoverflow question seem to answer my question. However, when I followed the solution mentioned, I get the following error

set_model() missing 1 required positional argument: 'model'

Here's my code to create a model:

model = Sequential()

model.add(Conv2D(64, (5, 5), input_shape=(IMG_HEIGHT, IMG_WIDTH, 3), activation='relu'))

model.add(Conv2D(64, (5, 5), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.2))

model.add(Conv2D(128, (5, 5), activation='relu'))
model.add(Conv2D(128, (5, 5), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.2))

model.add(Conv2D(256, (5, 5), activation='relu'))
model.add(Conv2D(256, (5, 5), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(BatchNormalization(axis=3))
model.add(Dropout(0.2))

model.add(Flatten())
model.add(Dense(256, activation='relu'))

model.add(Dropout(0.5))

model.add(Dense(256, activation='relu'))

model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

learning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', 
                                            patience=3, 
                                            verbose=1, 
                                            factor=0.4, 
                                            min_lr=0.0001)
csvlogger = CSVLogger("solution.csv", separator='\t')
checkpoint = ModelCheckpoint("models/best_model5.h5", monitor="val_acc", save_best_only=True, mode='max')
learning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', 
                                            patience=3, 
                                            verbose=1, 
                                            factor=0.4, 
                                            min_lr=0.00001)

class MyCallback(keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        lr = self.model.optimizer.lr
        decay = self.model.optimizer.decay
        iterations = self.model.optimizer.iterations
        lr_with_decay = lr / (1. + decay * K.cast(iterations, K.dtype(decay)))
        print(K.eval(lr_with_decay))

model.fit_generator(datagen.flow(x_train, y_train, batch_size=75), 
                           epochs=10, validation_data=(x_validation, y_test),verbose=1, 
                           steps_per_epoch=x_train.shape[0], callbacks=[csvlogger, checkpoint, MyCallback])

How do I get past this error "set_model() missing 1 required positional argument: 'model' " Below is the stack trace

TypeError                                 Traceback (most recent call last)
<ipython-input-12-1826a19039cd> in <module>()
    128 model.fit_generator(datagen.flow(x_train, y_train, batch_size=75), 
    129                            epochs=10, validation_data=(x_validation, y_test),verbose=1,
--> 130                            steps_per_epoch=x_train.shape[0], callbacks=[csvlogger, checkpoint, MyCallback])
    131 model.save('trained_model5.h5')
    132 

/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name +
     90                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/usr/local/lib/python3.6/dist-packages/keras/models.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1274                                         use_multiprocessing=use_multiprocessing,
   1275                                         shuffle=shuffle,
-> 1276                                         initial_epoch=initial_epoch)
   1277 
   1278     @interfaces.legacy_generator_methods_support

/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name +
     90                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   2131         else:
   2132             callback_model = self
-> 2133         callbacks.set_model(callback_model)
   2134         callbacks.set_params({
   2135             'epochs': epochs,

/usr/local/lib/python3.6/dist-packages/keras/callbacks.py in set_model(self, model)
     50     def set_model(self, model):
     51         for callback in self.callbacks:
---> 52             callback.set_model(model)
     53 
     54     def on_epoch_begin(self, epoch, logs=None):

TypeError: set_model() missing 1 required positional argument: 'model'

Also, another question that I have is, whether the above solution is correct.This tensorflow link about Adam Optimizer suggests learning rate to be calculated as:

lr_t <- learning_rate * sqrt(1 - beta2^t) / (1 - beta1^t)

This seems quite different from the solution that is mentioned in the other link. Did I miss something?

Answer 1

Actually, in the model.fit_generator method's callbacks parameter, you are passing the class instead of an object of that class.

It should be

my_calback_object = MyCallback() # create an object of the MyCallback class

model.fit_generator(datagen.flow(x_train, y_train, batch_size=75), 
                    epochs=10, validation_data=(x_validation, y_test),
                    verbose=1, steps_per_epoch=x_train.shape[0],
                    callbacks=[csvlogger, checkpoint, my_callback_object])

Answer 2

The error means you are not providing a value for the argument at position 1 (starting at 0) which is called "model". The caller is Tensorflow itself, so the fault is most likely not there.

This error suggests that you are calling the static method rather than the method on the object, thus passing only 1 argument instead of 2. That is because when you call a method on an object, the object as passed as the first argument and "model" will be passed as the second argument.

In short, your mistake is that your callback is a "class" and not an "object". Make sure you are providing an instance of your callback class and not the class itself.

Like this (notice the brackets after "MyCallback"):

model.fit_generator(datagen.flow(x_train, y_train, batch_size=75), epochs=10, validation_data=(x_validation, y_test),verbose=1, steps_per_epoch=x_train.shape[0], callbacks=[csvlogger, checkpoint, MyCallback()])

Answer 3

This is more of a comment, it just did not fit.

It is quite strange. The default implementation of set_model that MyCallback inherits is:

def set_model(self, model):
    self.model = model

and this is exactly how it is being called by according to the stack trace:

/usr/local/lib/python3.6/dist-packages/keras/callbacks.py in set_model(self, model)
     50     def set_model(self, model):
     51         for callback in self.callbacks:    
---> 52             callback.set_model(model)

My only guess at this point is that there is some version mismatch on your system. You might have some old .pyc lying around as well. I would try to debug this by editing /usr/local/lib/python3.6/dist-packages/keras/callbacks.py directly. For example, add a print statement before line 52 to make sure this code really runs. Then, drop into pdb (add import pdb; pdb.set_trace) and check why it is complaining. It is a basic python question at this point.