Keras - LeakyReLU has no attribute name error when saving model

Question

I'm using a LeakyReLU activation in my model. I'm able to train it. But when I train to save the model,

discriminator_model.save(os.path.join(output_folder_path, 'discriminator_model_{0}.h5'.format(iteration_no)))

I get the below error

AttributeError: 'LeakyReLU' object has no attribute '__name__'

I'm using keras-gpu 2.2.4 with tensorflow-gpu 1.12.0 backend. This is my model:

discriminator_model = Sequential()
discriminator_model.add(Conv2D(64, 5, strides=2, input_shape=(28, 28, 1), padding='same', activation=LeakyReLU(alpha=0.2)))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(128, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(256, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(512, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Flatten())
discriminator_model.add(Dense(1))
discriminator_model.add(Activation('sigmoid'))
discriminator_model.summary()

Initially, I was using

discriminator_model.add(Conv2D(128, 5, strides=2, padding='same', activation=LeakyReLU(alpha=0.2)))

But it was suggested here and here to add LeakyReLU as a separate activation layer. No luck even after trying that.

Full stack trace:

Traceback (most recent call last):
  File "/opt/PyCharm/pycharm-community-2018.3.3/helpers/pydev/pydevd.py", line 1741, in <module>
    main()
  File "/opt/PyCharm/pycharm-community-2018.3.3/helpers/pydev/pydevd.py", line 1735, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/opt/PyCharm/pycharm-community-2018.3.3/helpers/pydev/pydevd.py", line 1135, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/opt/PyCharm/pycharm-community-2018.3.3/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "..../Workspace/src/v01/MnistTrainer.py", line 100, in <module>
    main()
  File "..../Workspace/src/v01/MnistTrainer.py", line 92, in main
    mnist_trainer.train(train_steps=100, log_interval=1, save_interval=1)
  File "..../Workspace/src/v01/MnistTrainer.py", line 56, in train
    self.save_models(output_folder_path, i + 1)
  File "..../Workspace/src/v01/MnistTrainer.py", line 69, in save_models
    os.path.join(output_folder_path, 'discriminator_model_{0}.h5'.format(iteration_no)))
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/engine/network.py", line 1090, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/engine/saving.py", line 382, in save_model
    _serialize_model(model, f, include_optimizer)
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/engine/saving.py", line 83, in _serialize_model
    model_config['config'] = model.get_config()
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/engine/sequential.py", line 278, in get_config
    'config': layer.get_config()
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/layers/convolutional.py", line 493, in get_config
    config = super(Conv2D, self).get_config()
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/layers/convolutional.py", line 226, in get_config
    'activation': activations.serialize(self.activation),
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/activations.py", line 176, in serialize
    return activation.__name__
AttributeError: 'LeakyReLU' object has no attribute '__name__'

Answer 1

Edited part( Thanks @NagabhushanSN for mentioning the remaining issue)

There is a line of the code where we still have discriminator_model.add(Conv2D(64, 5, strides=2, input_shape=(28, 28, 1), padding='same', activation=LeakyReLU(alpha=0.2))) , it is the second line of the code.

If we modify that line, the final corrected code should be like this:

discriminator_model = Sequential()
discriminator_model.add(Conv2D(64, 5, strides=2, input_shape=(28, 28, 1), padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(128, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(256, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(512, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Flatten())
discriminator_model.add(Dense(1))
discriminator_model.add(Activation('sigmoid'))
discriminator_model.summary()

And this one should work well on the most recent version of tensroflow, I tested on 1.8.0 and it works fine. However, if check your code with older version like tesnorflow1.1.0, we get the same error.

For that case, I suggest update tensorflow to a higher version

• To check the current tensorflow version python is using, do as here.
• To update tensorflow, this post seems good enough to show how to do it.