I get "ValueError: HydraConfig was not set" when I use "hydra.initialize"

I want to use Hydra in Google Colab, and based on the official Hydra documentation (https://hydra.cc/docs/advanced/compose_api/) I do the following:

import hydra
from hydra.core.hydra_config import HydraConfig
from omegaconf import DictConfig, OmegaConf


def main(cfg: DictConfig) -> None:
    print(f"hydra.__version__ = {hydra.__version__}")
    print(OmegaConf.to_yaml(cfg))
    print(f"Output directory  : {hydra.core.hydra_config.HydraConfig.get().runtime.output_dir}")


if __name__ == "__main__":
    with hydra.initialize(version_base=None, config_path="conf"):
        cfg = hydra.compose(config_name="config")
        main(cfg)

In the same folder with the above code (which is in a file named test_hydra.py), I have a folder named "conf", with a file named "config.yaml", with a single property "my_test_hydra: test_hydra". The second "print" (from main) is correctly printing the content of the "conf/config.yaml". This explanation is about the local execution, to make easy for someone who would like to try it.

I get the following output:

hydra.version = 1.3.2
my_test_hydra: test_hydra

The problem is that I get the following error:

Traceback (most recent call last):
File "./test_hydra/test_hydra.py", line 17, in
main(cfg)
File "./test_hydra/test_hydra.py", line 11, in main
print(f"Output directory : {hydra.core.hydra_config.HydraConfig.get().runtime.output_dir}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./lib/python3.11/site-packages/hydra/core/hydra_config.py", line 31, in get
raise ValueError("HydraConfig was not set")
ValueError: HydraConfig was not set

Why is it HydraConfig not initialized? Using "@hydra.main(version_base=None, config_path="conf", config_name="config")" is not an option, because I am inside Google Colab. I tried the above mentioned code in Google Colab, and locally, but the result is the same error. Before asking here, I checked the official documentation, but I found nothing for this topic.

Running the Python code with Hydra, I was expecting to see the output directory, but I got an error. I tried to configure Hydra differently (for example with initialize_config_dir instead of initialize), but I got the same error. Again, the error is thrown not only when I execute the code in Google Colab, but also locally.

The Compose API is stateless and does not set the HydraConfig singleton, nor does manipulate the working directory, configuring logging or doing other things @hydra.main is doing.