Typesafe Config: best pattern for overriding substitutions in later settings?

Question

Ok, I have tried the following and it's not producing the effect that I'd have wished for. I have a reference.conf specifying filesystem locations relative to a prefix, which looks somewhat like this:

// reference.conf
myapp {
    // The dummy path makes the error message easy to diagnose
    s3.prefix = "s3://environment/variable/S3_PREFIX/missing"
    s3.prefix = ${?S3_PREFIX}

    file1 = ${myapp.s3.prefix}"/file1.csv"
    file2 = ${myapp.s3.prefix}"/file2.csv"
    // ...
}

And then I supply an application.conf file that looks more or less like this:

// application.conf
myapp.s3.prefix = "s3://some-bucket/some/path/to/the/files"
myapp.file2 = "s3://some-other-bucket/some/path/file2.csv"

Now, when my application does a ConfigFactory.load(), from :

1. Parse the reference.conf file, perform the substitutions, and generate a Config object where:
   • myapp.s3.prefix = "/environment/variable/S3_PREFIX/missing"
   • myapp.file1 = "/environment/variable/S3_PREFIX/missing/file1.csv"
   • myapp.file2 = "/environment/variable/S3_PREFIX/missing/file2.csv".
2. Parse the application.conf file and generate a Config object where:
   • library.prefix = "/some/path/to/the/files"
   • myapp.file2 = "s3://some-other-bucket/some/path/file2.csv".
3. Merge the two objects, with the reference.conf object as fallback. The resulting Config object therefore has:
   • library.prefix = "s3://some-bucket/some/path/to/the/files" (as in application.conf)
   • library.file1 = "s3://environment/variable/S3_PREFIX/missing/file1.csv" (as in reference.conf).
   • library.file2 = "s3://some-other-bucket/some/path/file2.csv" (as in application.conf).

But as you may have guessed from my example, what I'm trying to do is have the reference.conf specify default paths relative to a root directory, and allow any mix of both of these:

1. Override the default root directory from the application.conf, so that the default relative paths from the reference.conf are looked up from there.
2. Override the path for any individual file in the application.conf, so the application can use a path not in the root directory.

The only thing I've been able to come up with so far is this:

// reference.conf
myapp {
    // The dummy path makes the error message easy to diagnose
    s3.prefix = "s3:/environment/variable/S3_PREFIX/missing"
    s3.prefix = ${?S3_PREFIX}

    file1 = "file1.csv"
    file2 = "file2.csv"
    // ...
}

// application.conf
myapp.s3.prefix = "s3://some-bucket/some/path/to/the/files"
myapp.file2 = "s3://some-other-bucket/some/path/file2.csv"


// The Java code for using the config must append prefix and file location,
// and needs to have smarts about relative vs. absolute paths.

final Config CONF = ConfigFactory.load();

String getS3URLFor(String file) {
    String root = CONF.getString("myapp.s3.prefix");
    String path = CONF.getString("myapp." + file);
    return relativeVsAbsoluteSensitiveMerge(root, path);
}

String relativeVsAbsoluteSensitiveMerge(String root, String path) {
    if (isAbsoluteReference(path)) {
        return path; 
    } else {
        return root + "/" + path;
    }
}

boolean isAbsoluteReference(String path) {
   // ...
}

I don't really like this solution. Can anybody think of something better?

Answer 1

You can parse first and then resolve:

ConfigFactory.parseResources(s"file.conf")
  .withValue("myKey", ConfigValueFactory.fromAnyRef("newValue"))
  .resolve()