pythonpython-3.xyamlomegaconf

OmegaConf - Merging multiple files with name as key


Question

I am experimenting with the OmegaConf library and am having trouble with the OmegaConf.merge method. Referring to the project structure, there are two configuration files. When using the OmegaConf.merge method, it combines both the configurations into one as expected. However, this leaves me confused as I have a hard time determining which configurations belong to a specific configuration file.

The reason why I wish to merge the configurations is because I would like to have a single global variable with all the configurations to use across modules.

I do not wish to use the Hydra library due to limitations faced previously.

Project Structure

project_root
├── config
│   ├── example1.yaml
│   └── example2.yaml
└── src
    └── main.py

Code

config/example1.yaml

file: foods.txt
check: --A--

sample: XXA1

config/example2.yaml

items:
  - XXA1
  - XXA66
  
users:
  - user1
  - user2

src/main.py

import omegaconf
from omegaconf import OmegaConf

CONFIG = None  # Configurations stored here which will be imported across multiple modules.

example1_config = OmegaConf.load('../config/example1.yaml')
example2_config = OmegaConf.load('../config/example2.yaml')

CONFIG = OmegaConf.merge(example1_config, example2_config )

print(OmegaConf.to_yaml(merged_configs))

Current Output

file: foods.txt
check: --A--
sample: XXA1
items:
- XXA1
- XXA66
users:
- user1
- user2

Desired Output

example1:
  file: foods.txt
  check: --A--
  sample: XXA1
example2:
  items:
  - XXA1
  - XXA66
  users:
  - user1
  - user2

Solution

  • OmegaConf.merge combines the keys of its inputs, much like python's builtin dict.update method.

    To get your desired output, you'll need to add a level of nesting to your inputs:

    CONFIG = OmegaConf.merge(
        {"example1": example1_config},
        {"example2": example2_config},
    )