Hyperopt: Define parameter which is dependent on other parameter

Question

I am using python package hyperopt and I have a parameter a which requires to be larger than parameter b.

For example, I hope my parameter space is like

from hyperopt import hp

space = {"b": hp.uniform(0, 0.5), "a": hp.uniform(b, 0.5)}

Which, requires a to be at least larger than b, how can I do that?

Thanks in advance

Answer 1

A simple option is to use the ability of hyperopt to nest parameters. You can thus define a hyper-parameter space like you want:

space = hp.uniform("a", hp.uniform("b", 0, 0.5), 0.5)

Only "a"'s value is passed to the function that you optimize (because this is the hyper-parameter space), but hyperopt.fmin() will return both parameters.

A similar option, but where the function to be optimized receives both parameters is:

b_var = hp.uniform("b", 0, 0.5)
space = {"b": b_var, "a": hp.uniform("a", b_var, 0.5)}

Finally, it might be simpler to change a bit the inputs to the optimized function: parameter a can be replaced by a_fraction running between 0 and 1 and interpolating between b and 0.5 (i.e. a_fraction = 0 yields a = b and a_fraction = 1 gives a = 0.5 inside the modified function to be optimized). The parameter space thus has the usual form:

space = {"b": hp.uniform("b", 0, 0.5), "a_fraction": hp.uniform("a_fraction", 0, 1)}

There is an interesting discussion at https://github.com/hyperopt/hyperopt/issues/175#issuecomment-29401501.