What are hp.Discrete and hp.Realinterval? Can I include more values in hp.realinterval instead of just 2?

Question

I am using Hyperparameter using HParams Dashboard in Tensorflow 2.0-beta0 as suggested here https://www.tensorflow.org/tensorboard/r2/hyperparameter_tuning_with_hparams

I am confused in step 1, I could not find any better explanation. My questions are related to following lines:

HP_NUM_UNITS = hp.HParam('num_units', hp.Discrete([16, 32]))
HP_DROPOUT = hp.HParam('dropout', hp.RealInterval(0.1, 0.2))
HP_OPTIMIZER = hp.HParam('optimizer', hp.Discrete(['adam', 'sgd']))

My question: I want to try more dropout values instead of just two (0.1 and 0.2). If I write more values in it then it throws an error- 'maximum 2 arguments can be given'. I tried to look for documentation but could not find anything like from where these hp.Discrete and hp.RealInterval functions came. Any help would be appreciated. Thank you!

Answer 1

Good question. They notebook tutorial lacks in many aspects. At any rate, here is how you do it at a certain resolution res

for dropout_rate in tf.linspace(
      HP_DROPOUT.domain.min_value,
      HP_DROPOUT.domain.max_value,
      res,): 

Answer 2

By looking at the implementation to me it really doesn't seem to be GridSearch but MonteCarlo/Random search (note: this is not 100% correct, please see my edit below)

So on every iteration a random float of that real interval is chosen

If you want GridSearch behavior just use "Discrete". That way you can even mix and match GridSearch with Random search, pretty cool!

Edit: 27th of July '22: (based on the comment of @dpoiesz)

Just to make it a little more clear, as it is sampled from the intervals, concrete values are returned. Therefore, those are added to the grid dimension and grid search is performed using those