Using Hydra to Select Multiple Structured Configs

I have an existing hydra setup with yaml files, and am trying to migrate it to use structured configs to get the type annotation and inheritance advantages. One sticking point I am having is that part of my present yaml setup follows the select multiple configs from a config group pattern from the hydra docs. I am struggling to convert that pattern to use structured config classes instead of the yaml files. Here is my attempt to convert the exact example in the docs to structured configs:

from dataclasses import dataclass, field
from typing import Any

import hydra
from hydra.core.config_store import ConfigStore
from omegaconf import OmegaConf


@dataclass
class Amazon:
    domain: str = "amazon.com"


@dataclass
class Facebook:
    domain: str = "facebook.com"


@dataclass
class Google:
    domain: str = "google.com"


@dataclass
class Apache:
    host: str = "localhost"
    port: int = 443


@dataclass
class Server:
    site: list[Any] = field(default_factory=lambda: [Amazon, Facebook, Google])
    tech: Any = field(default_factory=Apache)


DEFAULTS = [{"server/site": ["facebook", "google"]}]


@dataclass
class Config:
    server: Any = field(default_factory=Server)
    defaults: list[Any] = field(default_factory=lambda: DEFAULTS)


@hydra.main(version_base=None, config_name="config")
def my_app(cfg: Config) -> None:
    print(OmegaConf.to_yaml(cfg))


if __name__ == "__main__":
    cs = ConfigStore.instance()
    cs.store(group="server/site", name="facebook", node=Facebook)
    cs.store(group="server/site", name="amazon", node=Amazon)
    cs.store(group="server/site", name="google", node=Google)
    cs.store(name="config", node=Config)
    my_app()

However, when I run that I get an error:

hydra.errors.ConfigCompositionException: In 'server/site/google': ValidationError raised while composing config:
Merge error: Google is not a subclass of Facebook. value: {'domain': 'google.com'}
    full_key: 
    object_type=dict

It seems like the default server/site being a list is making hydra try to merge the configs for facebook and google instead of having a list-of-dicts as the config. What am I doing wrong?

Solution

There are several issues with your port. If you look at the resulting config in the example page, you would notice that site is not a list but a dictionary. The different sites each have a different key hard coded into as a way avoid them filling the same node.

The example below creates a Site class with optional fields of each of the configs. This lines up better with the example you are porting (besides the fact that list composition is not supported).

In addition, it overrides the packages if the site configs to be in the correct package when storing the configs in the ConfigStore.

from dataclasses import dataclass, field
from typing import Any, Optional

import hydra
from hydra.core.config_store import ConfigStore
from omegaconf import OmegaConf


@dataclass
class Amazon:
    domain: str = "amazon.com"


@dataclass
class Facebook:
    domain: str = "facebook.com"


@dataclass
class Google:
    domain: str = "google.com"


@dataclass
class Apache:
    host: str = "localhost"
    port: int = 443


@dataclass
class Site:
    facebook: Optional[Facebook] = None
    google: Optional[Google] = None
    amazon: Optional[Amazon] = None


@dataclass
class Server:
    site: Site = field(default_factory=Site)
    tech: Any = field(default_factory=Apache)


DEFAULTS = ["_self_", {"server/site": ["facebook", "google"]}]


@dataclass
class Config:
    server: Any = field(default_factory=Server)
    defaults: list[Any] = field(default_factory=lambda: DEFAULTS)


@hydra.main(version_base=None, config_name="config")
def my_app(cfg: Config) -> None:
    print(OmegaConf.to_yaml(cfg))


if __name__ == "__main__":
    cs = ConfigStore.instance()
    cs.store(group="server/site", name="facebook", node=Facebook, package="server.site.facebook")
    cs.store(group="server/site", name="amazon", node=Amazon, package="server.site.amazon")
    cs.store(group="server/site", name="google", node=Google, package="server.site.google")
    cs.store(name="config", node=Config)
    my_app()