typesafe-confighocon

HOCON array substitution from envs


I am using the HOCON config format and parsing lib is from typesafe.Config. HOCON supports env vars injections and overrides. like:

my.config = "asd"
my.config = ${?MY_ENV_VAR} 

this will substitute the default value "asd" if there is a env var called MY_ENV_VAR presented. however I can't seem to find any good way to do list env substitution. like:

my.config = [1,2,3,4]
my.config = ${?MY_ENV_LIST}

because by default, env vars from outside will be default to string, so [1,2,3,4] will be considered as "[1,2,3,4]" from the library's perspective, therefore it can't be seen as a list and will produce runtime error like this:

com.typesafe.config.ConfigException$WrongType: application.conf: 5: application.boolliststring has type STRING rather than LIST

at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:133) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:145) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:151) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:159) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:164) at com.typesafe.config.impl.SimpleConfig.getList(SimpleConfig.java:212)

anyone has any idea what is the proper way to do list env substitution in HOCON format? thanks in advance.


Solution

  • The Answer

    Going from one environment variable to an array is not possible out of the box, as the other answer states.

    You could re-parse the string loaded from config:

    // application.conf
    my.config = "list = [1, 2, 3]"
    my.config = ${?LIST_VAR}
    
    // code
    String listString = ConfigFactory.load().getString("my.config")
    ConfigFactory.parseString(listString).getIntList("list")
    

    Then set LIST_VAR='list = [4, 5, 6]' to override the defaults. Note that you need list = because an object is required at the top level of hocon, you can't use an array.

    - or -

    If your data is clean enough, you could just split on ,s:

    // application.conf
    my.config = "foo,bar,baz"
    my.config = ${?CSV_VAR}
    
    // java code
    String csvString = ConfigFactory.load().getString("my.config")
    String[] parameters = csvString.split(",")
    

    Then, just set CSV_VAR=bing,bang,boom,pow (no []).

    Further Reading

    On the other hand, if you use separate environment variables for each value, there are several options.

    No Defaults

    The simplest, if you don't need defaults, looks like this:

    my.config = [ ${?MY_ENV_VAR}, ${?MY_ENV_VAR_TWO} ]
    

    Any values that are not defined are omitted.

    Adding to defaults

    If you only need to add to default values, you could use += syntax:

    my.config = [1, 2]
    my.config += ${?MY_ENV_VAR}
    my.config += ${?MY_ENV_VAR_TWO}
    

    Any values that are not defined are not added to the array.

    Maximum Flexibility

    The most flexible option I've found is to use positional syntax in your application.conf (or reference.conf or -D options or anywhere else you provide config):

    my.config.0 = 1                    // always set to 1  
    my.config.1 = 2                    // defaults to 2 if MY_ENV_VAR is not set
    my.config.1 = ${?MY_ENV_VAR}
    my.config.2 = ${?MY_ENV_VAR_TWO}   // totally optional
    my.config.3 = ${MY_ENV_VAR_THREE}  // doesn't have ?, so it is required
    

    Any values that are defined will be included, any that are not will be skipped over. For example, if MY_ENV_VAR=4, MY_ENV_VAR_THREE=6, and MY_ENV_VAR_TWO is not set, the resulting list will be [1, 4, 6].

    List of objects

    You can even define objects within the list, like this:

    my.nested.0.myField = 1
    my.nested.0.otherField = "hello"
    my.nested.1.myField = 2
    my.nested.1.myField = ${?MY_INT}
    my.nested.1.otherField = "goodbye"
    my.nested.1.otherField = ${?MY_STRING}
    my.nested.2.myField = ${OTHER_INT}       // Note lack of ?
    my.nested.2.otherField = ${OTHER_STRING} // Note lack of ?
    

    One catch with a list of config objects, at least in my testing, is all items need to be completely defined. That is why the fields that don't have defaults are required substitutions. If MY_INT=99, MY_STRING is not set, OTHER_INT=100, and OTHER_STRING=foo, the above renders to:

    other {
      nested = [
        { myField = 1, otherField = "hello" },
        { myField = 99, otherField = "goodbye" },
        { myField = 100, otherField = "foo" }
      ]
    }