Table of Contents
· Part 1: What is OmegaConf?
· Part 2: OmegaConf In Details
· 2.1 DictConfig
· 2.2 Interpolation
· 2.3 Merging Multiple YAMLs
· 2.4 Config Flags
· 2.5 Structured Configs
· Reference
Part 1: What is OmegaConf?
Introduction
OmegaConf is a YAML based hierarchical configuration system. It supports the following functions regarding to YAML:
- reading/writing YAML file as a well-designed class
DictConfig
- interpolations
- merging several YAML files
- command line overrides
Installation
pip install omegaconf
Part 2: OmegaConf In Details
2.1 DictConfig
Creation
from omegaconf import OmegaConf
conf = OmegaConf.create()
conf is the DictConfig
class object, and we can create it from
- dictionary
OmegaConf.create()
- list
OmegaConf.create()
- YAML file
OmegaConf.load()
- YAML string
OmegaConf.create()
- command line
sys.argv=['script.py', 'srv.port=82']
andOmegaConf.from_cli()
- dataclasses
OmegaConf.structured()
Correspondence
- If the required key is not available, None will return. In the above example,
conf.my_name==None
. Like Dict in Python, we can use get to given missing key a value. For example,conf.get('missing', 'new_value')
, thenconf.missing=='new_value'
Like Dict in Python, we can add a new key (a new dictionary) ???
means that the value must be set before it is called.
name:
first_name:???conf = OmegaConf.load('must_set.yaml')
print(conf)
conf.name.first_name="tompas"
print(conf)
- OmegaConf.to_yaml is used to transform
DictConfig
tostr
. - Reading and writing YAML file
# reading
from omegaconf import OmegaConf
conf = OmegaConf.load('ab.yaml')# writing
OmegaConf.save(config=conf, f="file_name_to_save")
2.2 Interpolation
config.yaml
server:
host: localhost
port: 80
client:
url: http://${server.host}:${server.port}/
server_port: ${server.port}
Code
conf = OmegaConf.load('config.yaml')
print(conf)
conf = OmegaConf.create(OmegaConf.to_yaml(conf, resolve=True))
print(conf)
Output
{'server': {'host': 'localhost', 'port': 80}, 'client': {'url': 'http://${server.host}:${server.port}/', 'server_port': '${server.port}'}}{'server': {'host': 'localhost', 'port': 80}, 'client': {'url': 'http://localhost:80/', 'server_port': 80}}
Other types of interpolation
(1) env resolvers
# env.yaml
user:
name: ${env:USER}
home: /home/${env:USER}# codes
conf = OmegaConf.load('env.yaml')
print(conf)
import os
os.environ["USER"]="TOM"
conf = OmegaConf.create(OmegaConf.to_yaml(conf, resolve=True))
print(conf)
(2) customer resolver
conf = OmegaConf.load('custm.yaml')
print(conf)
func=lambda x:int(x)+100
OmegaConf.register_resolver('value_may_change', func)
conf = OmegaConf.load('custm.yaml')
conf = OmegaConf.create(OmegaConf.to_yaml(conf, resolve=True))
print(conf)
Both env resolver and customer resolver needs to be set before they can be used.
2.3 Merging Multiple YAMLs
Codes
conf=OmegaConf.merge(conf_1, conf_2, conf_3)
Example
2.4 Config Flags
(1) make DictConfig
behave more like a regular class, and we can temporally change it
conf = OmegaConf.create({"a":[23, 45]})
conf.b=[1,2]
print(conf)OmegaConf.set_struct(conf, True)
from omegaconf import OmegaConf, open_dict
with open_dict(conf):
conf.c=[7,8]
print(conf)
(2) read-only flag
2.5 Structured Configs
Structured Configs is an enhanced version of dataclass
from Python package.
(1) Primitive types it supports
int, float, bool, str, Enums
from dataclasses import dataclass
from enum import Enum
class Height(Enum):
SHORT=0
TALL=1
@dataclass
class SimpleType:
num: int=10
val_f: float=3.14
is_awesome: bool=True
description: str="text"
h: Height=Height.TALLconf = OmegaConf.create(
SimpleType(num=200)
)
print(conf)
The difference between dataclass from Python with DictConfig is that DictConfig
will perform type checking or type conversion.
(2) Nesting
@dataclass
class Wheel:
radius:float=3@dataclass
class CircusBike:
front: Wheel=Wheel(radius=10)
rear: Wheel=Wheel(radius=20)
conf = OmegaConf.create(CircusBike)
print(conf)
Reference
Official Website