Specify Hydra multirun sweeps in a config file

Question

I would like to run a Hydra multirun, but specify the sweeps in a config file. I would like to know if there is a way to do this before asking for a feature request.

So far what I have tried is the following:

Tree structure:

.
├── conf
│   ├── compile
│   │   ├── base.yaml
│   │   └── grid_search.yaml
│   └── config.yaml
└── my_app.py

Content of my_appy.py:

import hydra
from omegaconf import DictConfig, OmegaConf

@hydra.main(config_path="conf", config_name="config")
def my_app(cfg : DictConfig) -> None:
    print(OmegaConf.to_yaml(cfg, resolve=True))

if __name__ == "__main__":
    my_app()

Content of conf/config.yaml:

defaults:
  - compile: base

Content of conf/compile/base.yaml:

loss: mse
optimizer: adam

Content of conf/compile/grid_search.yaml:

defaults:
  - base

lr: 1e-2,1e-3,1e-4

When I run python my_app.py -m compile=grid_search, I get the following output:

[2022-01-07 10:08:05,414][HYDRA] Launching 1 jobs locally
[2022-01-07 10:08:05,414][HYDRA]        #0 : compile=grid_search
compile:
  loss: mse
  optimizer: adam
  lr: 1e-2,1e-3,1e-4

This is an output I understand, because in this example there is no way to tell the difference between a config variable holding a list, and a config variable over which you want to sweep. Is there to indicate such a thing in a config file?

Basically I would like to be able to specify my grid searches in config files rather than in the command line or in shell scripts.

Bonus question: how would this be done for a sweep specified by a dictionary override, as in this issue?

Answer 1

I had also asked this question on GitHub, and got an answer. In conf/experiment/grid_search.yaml, you can have:

# @package _global_
hydra:
  sweeper:
    params:
      +compile.lr: 1e-2,1e-3,1e-4

Then you can run:

python my_app.py -m +experiment=grid_search

In order to have a sweep defined over a dictionary (the bonus part of my question), you can replace the final configuration line by:

+compile: "{lr:1e-2,wd:1e-4},{lr:1e-3,wd:1e-5}"

One big caveat: this will only be available in Hydra version 1.2! (pip install --upgrade hydra-core==1.2.0.dev2)

Answer 2

Even better try this in Hydra 1.2. Just add the following to your config file.

hydra:
  mode: MULTIRUN
  sweeper:
    params:
      +<PARAM_NAME>: <PARAM>

Here is a complete example:

network:
    inputs: 2
    outputs: 3
    layer_size: 60
    nr_layers: 8

optimiser:
    lr: 1e-3
    scheduler: exponential_lr
    iter: 5000
    
hydra:
    mode: MULTIRUN
    sweeper:
        params:
          +n: 5,10,15
          +a_lower: 0.5, 0.7, 0.9
          +a_upper: 1.1, 1.15
          +a: 0.91, 1.05, 1.09
          +pde_coeff: 0.1, 1.0

Now put the entire Python file as usual:

@hydra.main(version_base="1.2", config_path="saved_data", config_name="conf")
def main(cfg: DictConfig) -> None:

And run the python file without any argument.

python train.py