Python Hydra

Part 1: Project Configuration

There are several ways to configure projects:

• Command-line arguments

• configure file

{
    "mysql":{
        "host":"localhost",
        "user":"root",
        "passwd":"my secret password",
        "db":"write-math"
    },
    "other":{
        "preprocessing_queue":[
            "preprocessing.scale_and_center",
            "preprocessing.dot_reduction",
            "preprocessing.connect_lines"
            ],
        "use_anonymous":true
    }
}

Part 2: Basic Idea behind Hydra

• composite configuration files: each configuration file is used for one aspect of the project; compose the configuration just like composing code with Hydra
• dynamically create a hierarchical configuration by composition and override it through config files and the command line

Part 3: Hydra Components

3.1 compose function

The Compose API is useful when @hydra.main() is not applicable. For example:

• If you want to compose multiple configuration objects (Example with Ray): write out the default setting

results = []    
for model in ["alexnet", "resnet"]:        
   for dataset in ["cifar10", "imagenet"]:            
       overrides = [f"dataset={dataset}", f"model={model}"]   
       run_cfg = compose(overrides=overrides)            
       ret = train.remote(overrides, run_cfg)  
       results.append(ret)

• In parts of your application that does not have access to the command line

from omegaconf import OmegaConf 
from hydra.experimental import compose, initialize 
if __name__ == "__main__":    
   # initialize the Hydra subsystem.    
   # This is needed for apps that cannot have a standard 
   # @hydra.main() entry point    
   initialize(config_path="conf")    
   cfg = compose("config.yaml", 
            overrides=["db=mysql", "db.user=${env:USER}"]) 
   print(OmegaConf.to_yaml(cfg, resolve=True))

• unit test (example)

def test_with_initialize() -> None:
  with initialize(config_path="../hydra_app/conf"):
   # config is relative to a module
   cfg = compose(config_name="config", 
            overrides=["app.user=test_user"])

• jupyter notebook (example)

with initialize(config_path="cloud_app/conf"):
    cfg = compose(overrides=["+db=mysql"])
    print(cfg)

3.2 Reuse configuration module

defaults:
  - db@source: mysql
  - db@destination: mysql

3.3 Instantiate

configuration.yaml

trainer:  
   _target_: my_app.Trainer  
   optimizer:    
     _target_: my_app.Optimizer    
     algo: SGD    
     lr: 0.01  
   dataset:    
     _target_: my_app.Dataset    
     name: Imagenet    
     path: /datasets/imagenet

codes

class Optimizer:    
  algo: str    
  lr: float     
  def __init__(self, algo: str, lr: float) -> None:        
      self.algo = algo        
      self.lr = lr     
  def __repr__(self) -> str:        
      return f"Optimizer(algo={self.algo},lr={self.lr})"
@hydra.main(config_name="config")
def my_app(cfg: DictConfig) -> None:    
   optimizer = instantiate(cfg.trainer.optimizer)  
   print(optimizer)

Examples

• my_app.py

Reference

Blog

• Hydra — A fresh look at configuration for machine learning projects
• Keeping Up with PyTorch Lightning and Hydra
• How to Track Hyperparameters of Machine Learning Models?