Python OmegaConf

3 min readJan 10, 2021

Table of Contents

· Part 1: What is OmegaConf?
· Part 2: OmegaConf In Details
· 2.1 DictConfig
· 2.2 Interpolation
· 2.3 Merging Multiple YAMLs
· 2.4 Config Flags
· 2.5 Structured Configs
· Reference

Part 1: What is OmegaConf?

Introduction

OmegaConf is a YAML based hierarchical configuration system. It supports the following functions regarding to YAML:

reading/writing YAML file as a well-designed class DictConfig
interpolations
merging several YAML files
command line overrides

Installation

pip install omegaconf

Part 2: OmegaConf In Details

2.1 `DictConfig`

Creation

from omegaconf import OmegaConf
conf = OmegaConf.create()

conf is the DictConfig class object, and we can create it from

dictionary OmegaConf.create()
list OmegaConf.create()
YAML file OmegaConf.load()
YAML string OmegaConf.create()
command line sys.argv=['script.py', 'srv.port=82'] and OmegaConf.from_cli()
dataclasses OmegaConf.structured()

Correspondence

If the required key is not available, None will return. In the above example, conf.my_name==None . Like Dict in Python, we can use get to given missing key a value. For example, conf.get('missing', 'new_value'), then conf.missing=='new_value' Like Dict in Python, we can add a new key (a new dictionary)
??? means that the value must be set before it is called.

name:
    first_name:???conf = OmegaConf.load('must_set.yaml')
print(conf)
conf.name.first_name="tompas"
print(conf)

OmegaConf.to_yaml is used to transform DictConfigto str.
Reading and writing YAML file

# reading
from omegaconf import OmegaConf
conf = OmegaConf.load('ab.yaml')# writing 
OmegaConf.save(config=conf, f="file_name_to_save")

2.2 `Interpolation`

config.yaml

server:
  host: localhost
  port: 80
client:
  url: http://${server.host}:${server.port}/
  server_port: ${server.port}

Code

conf = OmegaConf.load('config.yaml')
print(conf)
conf = OmegaConf.create(OmegaConf.to_yaml(conf, resolve=True))
print(conf)

Output

{'server': {'host': 'localhost', 'port': 80}, 'client': {'url': 'http://${server.host}:${server.port}/', 'server_port': '${server.port}'}}{'server': {'host': 'localhost', 'port': 80}, 'client': {'url': 'http://localhost:80/', 'server_port': 80}}

Other types of interpolation

(1) env resolvers

# env.yaml
user:
  name: ${env:USER}
  home: /home/${env:USER}# codes
conf = OmegaConf.load('env.yaml')
print(conf)
import os 
os.environ["USER"]="TOM"
conf = OmegaConf.create(OmegaConf.to_yaml(conf, resolve=True))
print(conf)

(2) customer resolver

conf = OmegaConf.load('custm.yaml')
print(conf)
func=lambda x:int(x)+100
OmegaConf.register_resolver('value_may_change', func)
conf = OmegaConf.load('custm.yaml')
conf = OmegaConf.create(OmegaConf.to_yaml(conf, resolve=True))
print(conf)

Both env resolver and customer resolver needs to be set before they can be used.

2.3 Merging Multiple YAMLs

Codes

conf=OmegaConf.merge(conf_1, conf_2, conf_3)

Example

2.4 Config Flags

(1) make DictConfig behave more like a regular class, and we can temporally change it

conf = OmegaConf.create({"a":[23, 45]})
conf.b=[1,2]
print(conf)OmegaConf.set_struct(conf, True)
from omegaconf import OmegaConf, open_dict
with open_dict(conf):
    conf.c=[7,8]
print(conf)

(2) read-only flag

2.5 Structured Configs

Structured Configs is an enhanced version of dataclass from Python package.

(1) Primitive types it supports

int, float, bool, str, Enums

from dataclasses import dataclass
from enum import Enum
class Height(Enum):
    SHORT=0
    TALL=1
@dataclass
class SimpleType:
   num: int=10
   val_f: float=3.14
   is_awesome: bool=True
   description: str="text"
   h: Height=Height.TALLconf = OmegaConf.create(
   SimpleType(num=200)
)
print(conf)

The difference between dataclass from Python with DictConfig is that DictConfig will perform type checking or type conversion.

(2) Nesting

@dataclass
class Wheel:
  radius:float=3@dataclass
class CircusBike:
    front: Wheel=Wheel(radius=10)
    rear: Wheel=Wheel(radius=20)
        
conf = OmegaConf.create(CircusBike)
print(conf)

Reference

Official Website