How to override hydra working dir from within a script?

Question

I know that I can change the working dir in config by setting hydra.run.dir=XXX from the command line. But how to do it properly from script w/o using CLI arguments in a way that even the logs are saved in the dir which I set?

This code won't work because:

1. the hydra and its loggers are already initialized when I try to change the dir and
2. there is no such attribute cfg.hydra.

UPD: I got a pointer in the comments. I could change the hydra parameters in the block if __name__ == 'main': before hydra is called. But how to get access and modify hydra.run.dir from the script?

    @hydra.main(config_path="conf", config_name="config")
    def main(cfg):
        cfg.hydra.run.dir = "./c_out/cached_loss"  # no such attribute
        logger.info('I log something')

My hydra config looks like this:

defaults:                     
  - hydra/job_logging: custom_logging 

# hydra/custom_logging.yaml
# python logging configuration for tasks                           
version: 1                                                         
formatters:                                                        
  simple:                                                          
    format: '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s' 
handlers:                                                          
  console:                                                         
    class: logging.StreamHandler                                   
    formatter: simple                                              
    stream: ext://sys.stdout                                       
  file:                                                            
    class: logging.FileHandler                                     
    formatter: simple                                              
    # relative to the job log directory                            
    filename: ${hydra.job.name}.log                                
root:                                                              
  level: INFO                                                      
  handlers: [console, file]                                        
                                                                   
disable_existing_loggers: false                                    

Answer 1

The @hydra.main decorator reads the command line arguments from sys.argv and creates the output directory and sets up logging based on the arguments, before the decorated function is executed. You don't have the configuration before entering the function, but you could add the hydra.run.dir=XXX command line argument before calling the function with this kind of hack:

@hydra.main(config_path="conf", config_name="config")
def main(cfg):
    logger.info('I log something')

if __name__ == 'main':
    sys.argv.append('hydra.run.dir=c_out/cached_loss')
    main()

Answer 2

This can be achieved via omegaconf interpolation
For example my use case when I create directory named with uuid
First we register resolver with the function we need instead of lambda

from omegaconf import OmegaConf

OmegaConf.register_resolver("uuid", lambda : "fdjsfas-3213-kjfdsf")

in hydra config

hydra:
  run:
    dir: ./outputs/training/${uuid:}

This is still not really accesing from script but it allows python code to generate config variables. I don't really think there is a normal way to alter the hydra config after it was initialized.

P.S. I use structured configs and had to change the code so it may not actually work but I hope you got the idea

Answer 3

You can change it BEFORE the script starts by overriding that parameter.

python foo.py hydra.run.dir=something

You can also change it in your config: config.yaml

hydra:
  run:
    dir: whatever

This can also use an environment variable in the config using OmegaConf env resolver.

hydra:
  run:
    dir: ${env:HYDRA_OUTPUT_DIR,default_output_dir}

if you just want to change the working directory at runtime you can do it with os.chdir()