Hparams plugin with tf.keras (tensorflow 2.0)

Question

I try to follow the example from the tensorflow docs and setup hyperparameter logging. It also mentions that, if you use tf.keras, you can just use the callback hp.KerasCallback(logdir, hparams). However, if I use the callback I don't get my metrics (only the outcome).

Answer 1

The trick is to define the Hparams config with the path in which TensorBoard saves its validation logs.

So, if your TensorBoard callback is set up as:

log_dir = 'path/to/training-logs'
tensorboard_cb = TensorBoard(log_dir=log_dir)

Then you should set up Hparams like this:

hparams_dir = os.path.join(log_dir, 'validation')

with tf.summary.create_file_writer(hparams_dir).as_default():
    hp.hparams_config(
        hparams=HPARAMS,
        metrics=[hp.Metric('epoch_accuracy')]  # metric saved by tensorboard_cb
    )

hparams_cb = hp.KerasCallback(
    writer=hparams_dir,
    hparams=HPARAMS
)

Answer 2

I managed but not entirely sure what was the magic word. Here my flow in case it helps.

callbacks.append(hp.KerasCallback(log_dir, hparams))

HP_NUM_LATENT = hp.HParam('num_latent_dim', hp.Discrete([2, 5, 100])) 
hparams = {
   HP_NUM_LATENT: num_latent,
}

model = create_simple_model(latent_dim=hparams[HP_NUM_LATENT])  # returns compiled model
model.fit(x, y, validation_data=validation_data, 
          epochs=4,
          verbose=2,
          callbacks=callbacks)