HOCON array substitution from envs

Question

I am using the HOCON config format and parsing lib is from typesafe.Config. HOCON supports env vars injections and overrides. like:

my.config = "asd"
my.config = ${?MY_ENV_VAR} 

this will substitute the default value "asd" if there is a env var called MY_ENV_VAR presented. however I can't seem to find any good way to do list env substitution. like:

my.config = [1,2,3,4]
my.config = ${?MY_ENV_LIST}

because by default, env vars from outside will be default to string, so [1,2,3,4] will be considered as "[1,2,3,4]" from the library's perspective, therefore it can't be seen as a list and will produce runtime error like this:

com.typesafe.config.ConfigException$WrongType: application.conf: 5: application.boolliststring has type STRING rather than LIST

at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:133) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:145) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:151) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:159) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:164) at com.typesafe.config.impl.SimpleConfig.getList(SimpleConfig.java:212)

anyone has any idea what is the proper way to do list env substitution in HOCON format? thanks in advance.

Answer 1

The Answer

Going from one environment variable to an array is not possible out of the box, as the other answer states.

You could re-parse the string loaded from config:

// application.conf
my.config = "list = [1, 2, 3]"
my.config = ${?LIST_VAR}

// code
String listString = ConfigFactory.load().getString("my.config")
ConfigFactory.parseString(listString).getIntList("list")

Then set LIST_VAR='list = [4, 5, 6]' to override the defaults. Note that you need list = because an object is required at the top level of hocon, you can't use an array.

- or -

If your data is clean enough, you could just split on ,s:

// application.conf
my.config = "foo,bar,baz"
my.config = ${?CSV_VAR}

// java code
String csvString = ConfigFactory.load().getString("my.config")
String[] parameters = csvString.split(",")

Then, just set CSV_VAR=bing,bang,boom,pow (no []).

Further Reading

On the other hand, if you use separate environment variables for each value, there are several options.

No Defaults

The simplest, if you don't need defaults, looks like this:

my.config = [ ${?MY_ENV_VAR}, ${?MY_ENV_VAR_TWO} ]

Any values that are not defined are omitted.

Adding to defaults

If you only need to add to default values, you could use += syntax:

my.config = [1, 2]
my.config += ${?MY_ENV_VAR}
my.config += ${?MY_ENV_VAR_TWO}

Any values that are not defined are not added to the array.

Maximum Flexibility

The most flexible option I've found is to use positional syntax in your application.conf (or reference.conf or -D options or anywhere else you provide config):

my.config.0 = 1                    // always set to 1  
my.config.1 = 2                    // defaults to 2 if MY_ENV_VAR is not set
my.config.1 = ${?MY_ENV_VAR}
my.config.2 = ${?MY_ENV_VAR_TWO}   // totally optional
my.config.3 = ${MY_ENV_VAR_THREE}  // doesn't have ?, so it is required

Any values that are defined will be included, any that are not will be skipped over. For example, if MY_ENV_VAR=4, MY_ENV_VAR_THREE=6, and MY_ENV_VAR_TWO is not set, the resulting list will be [1, 4, 6].

List of objects

You can even define objects within the list, like this:

my.nested.0.myField = 1
my.nested.0.otherField = "hello"
my.nested.1.myField = 2
my.nested.1.myField = ${?MY_INT}
my.nested.1.otherField = "goodbye"
my.nested.1.otherField = ${?MY_STRING}
my.nested.2.myField = ${OTHER_INT}       // Note lack of ?
my.nested.2.otherField = ${OTHER_STRING} // Note lack of ?

One catch with a list of config objects, at least in my testing, is all items need to be completely defined. That is why the fields that don't have defaults are required substitutions. If MY_INT=99, MY_STRING is not set, OTHER_INT=100, and OTHER_STRING=foo, the above renders to:

other {
  nested = [
    { myField = 1, otherField = "hello" },
    { myField = 99, otherField = "goodbye" },
    { myField = 100, otherField = "foo" }
  ]
}