pythoncommand-line-interfacefb-hydratyper

Using Typer and Hydra together


I have a simple Typer application:

import typer

app = typer.Typer()

@app.command()
def say_hi():
    print("Hi")

@app.callback()
def main():
    pass

if __name__ == "__main__":
    app()

I would like to use Hydra for the configuration management of the app, however I am not sure how to do that without losing the ability to override the config from the CLI.

My first attempt was:

import hydra
import typer
from omegaconf import DictConfig, OmegaConf

app = typer.Typer()

@app.command()
def say_hi():
    print("Hi")

@app.callback()
@hydra.main(config_path="conf", config_name="config")
def main(cfg: DictConfig) -> None:
    print(OmegaConf.to_yaml(cfg))

if __name__ == "__main__":
    app()

But I get an error saying:

RuntimeError: Type not yet supported: <class 'omegaconf.dictconfig.DictConfig'>

If I remove the DictConfig type annotation I get an error that cfg is missing.

I saw in Hydra docs the Compose API that allows to initialize the config without decorators:

@app.callback()
def main() -> None:
    with initialize(config_path="conf", job_name="test_app"):
        cfg = compose(config_name="config")
        print(OmegaConf.to_yaml(cfg))

but It seems that I can't override config from the command line in this case as those values are not recognized by the Typer app.

Any recommendations how it can be resolved?


Solution

  • The compose function accepts an optional list of override strings:

    with initialize(config_path="conf", job_name="test_app"):
        cfg = compose(config_name="config", overrides=["db=mysql", "db.user=me"])
    

    You will need to get a list of override strings from the command line, then pass that list to compose. Here is an example using Typer; similar patterns could work for e.g. argparse or click. (Disclaimer: I am not a Typer expert)

    from typing import List, Optional
    
    import typer
    from omegaconf import OmegaConf, DictConfig
    
    from hydra import compose, initialize
    
    app = typer.Typer()
    
    def my_compose(overrides: Optional[List[str]]) -> DictConfig:
        with initialize(config_path="conf", job_name="test_app"):
            return compose(config_name="config", overrides=overrides)
    
    @app.command()
    def say_hi(overrides: Optional[List[str]] = typer.Argument(None)):
        print("HI!")
        print(f"Got {overrides=}")
        cfg = my_compose(overrides)
        print("\nHydra config:")
        print(OmegaConf.to_yaml(cfg))
    
    @app.command()
    def say_bye(overrides: Optional[List[str]] = typer.Argument(None)):
        cfg = my_compose(overrides)
        ...
        print("BYE!")
    
    if __name__ == "__main__":
        app()
    
    $ python my_app.py say-hi +foo=bar +baz=qux
    HI!
    Got overrides=('+foo=bar', '+baz=qux')
    
    Hydra config:
    foo: bar
    baz: qux
    
    $ python my_app.py say-bye
    BYE!