Using Apache Commons Configuration to do variable interpolation, i.e. ${variable}, with a list of values in a properties file

Question

I am using Apache Commons Configuration to read a properties file, and I am perfectly able to do variable interpolation, and also to retrieve multi-valued properties as a list. However, I haven't been able to correctly load a property which has several values, where one of them is a reference (variable interpolation) to another multi-valued property.

Here's an example of my properties file (I have also tried using comma-separated syntax):

doc.mime=application/msword
doc.mime=application/vnd.openxmlformats-officedocument.wordprocessingml.document
doc.mime=${office.mime}

office.mime=application/x-tika-msoffice
office.mime=application/x-tika-ooxml

And how I read from it:

Configuration config = new PropertiesConfiguration("myFile");
final String[] mimesArray = config.getStringArray("doc.mime");
for(String mime : mimesArray) System.out.println(mime);
final List<Object> mimesList = config.getList("doc.mime");
System.out.println(mimesList);

This is the content I get with either method (getStringArray and getList):

[application/msword, application/vnd.openxmlformats-officedocument.wordprocessingml.document, application/x-tika-msoffice]

This is different from what I expected: the complete contents of both doc.mime and office.mime

Does anyone know if it is possible to interpolate the whole list of values in my other list? And if so, how is it done?

Answer 1

What Commons Configuration does

As you found out: When interpolating a multi-valued property Commons Configuration will only resolve the first value of that property. See the code at AbstractConfiguration#resolveContainerStore() line 1177.

I found some related issues:

CONFIGURATION-28: Someone wants (and gets) the exact opposite of what you want: Only the first value in a multi-valued property.

CONFIGURATION-55: More discussion about interpolation of multi-valued properties:

there is probably no correct solution to this problem because the expected results strongly depend on a concrete use case

Workaround: Combine the two lists in code

Definitely easier than customizing interpolation:

List<Object> mimesList = config.getList("doc.mime");
List<Object> officeList = config.getList("office.mime");
mimesList.addAll(officeList);
System.out.println(mimesList);

Raise this issue with the Commons Configuration project

Changing the whole variable interpolation system is probably difficult. But they could at least clarify the documentation.

Answer 2

Well, I did need this feature to : associate severals values to properties and be able to retrieve them all even by interpolation.

So I wrote a custom interpolation class and a custom properties configuration. Works well even with values with backslash and the default delimiter.

So here the custom interpolation class:

CustomInterPolation class

public class CustomInterpolation extends StrLookup {

    /*
     * (non-Javadoc)
     * 
     * @see org.apache.commons.lang.text.StrLookup#lookup(java.lang.String)
     */
    @Override
    public String lookup(String arg0) {

        String result = null;

        // Get the default delimiter.
        String delimiter = ""
                + PropertiesConfiguration.getDefaultListDelimiter();

        try {

            // Load the properties file.
            Configuration config = new PropertiesConfiguration(
                    "ressources/macro.properties");

            if (config.containsKey(arg0)) {

                // Get all values associated with the propertie.
                ArrayList<Object> values = (ArrayList<Object>) config
                        .getList(arg0);

                StringBuilder strBuild = new StringBuilder();
                Iterator<Object> itr = values.iterator();

                while (itr.hasNext()) {

                    // Append the property to the string.
                    strBuild.append((String) itr.next());

                    if (itr.hasNext()) {
                        // Adds the delimiter and backslash in order to retrieve
                        // all properties later.
                        strBuild.append("\\" + delimiter);
                    }

                }
                result = strBuild.toString();
            }

        } catch (ConfigurationException e) {
            // Nothing to do here...
        }
        // return null or all values concatenated
        return result;
    }

}

Now, in order to use correctly this customInterpolation class, we need to use a custom properties configuration:

CustomPropertiesConfiguration class

/**
 * The Class CustomPropertiesConfiguration.
 */
public class CustomPropertiesConfiguration extends PropertiesConfiguration {

    private String delimiter;

    /**
     * Instantiates a new custom properties configuration.
     */
    public CustomPropertiesConfiguration() {
        super();
        delimiter = PropertiesConfiguration.getDefaultListDelimiter()
                + "";
    }

    /**
     * Instantiates a new custom properties configuration.
     *
     * @param file the file
     * @throws ConfigurationException the configuration exception
     */
    public CustomPropertiesConfiguration (File file) throws ConfigurationException{
        super(file);
        delimiter = PropertiesConfiguration.getDefaultListDelimiter()
                + "";
    }

    /**
     * Instantiates a new custom properties configuration.
     *
     * @param fileName the file name
     * @throws ConfigurationException the configuration exception
     */
    public CustomPropertiesConfiguration(String fileName) throws ConfigurationException {
        super(fileName);
        delimiter = PropertiesConfiguration.getDefaultListDelimiter()
                + "";
    }

    /**
     * Instantiates a new custom properties configuration.
     *
     * @param url the url
     * @throws ConfigurationException the configuration exception
     */
    public CustomPropertiesConfiguration(URL url) throws ConfigurationException{
        super(url);
        delimiter = PropertiesConfiguration.getDefaultListDelimiter()
                + "";
    }

    /* (non-Javadoc)
     * @see org.apache.commons.configuration.AbstractConfiguration#getList(java.lang.String)
     */
    @Override
    public List<Object> getList(String key) {

        // Get the list of values associated with the property
        // Implicit call to the custom interpolation class
        List<Object> properties = super.getList(key);


        ArrayList<Object> extendedProperties = new ArrayList<Object>();

        Iterator<Object> itrProperties = properties.iterator();
        // Go through all properties and retrieve values concatenated by the custom interpolation
        while (itrProperties.hasNext()) {

            String propertie = (String) itrProperties.next();

            if (propertie.contains(delimiter)) {

                //Split concatenated values.
                String[] extendedPropertiesTab = propertie.split("\\\\"+delimiter);

                // Add the retrieved values to the list of values.
                for (int i = 0; i< extendedPropertiesTab.length; ++i){
                    extendedProperties.add(extendedPropertiesTab[i]);
                }

            } else {
                extendedProperties.add(propertie);
            }

        }
        return extendedProperties;
    }

} 

And here, a small main class:

public class TestMacro {

    /**
     * @param args
     */
    public static void main(String[] args) {

        // Load properties file :

        try {

            // Add an interpolation to the configuration.
            // The string "custom" will be used to find value to interpolate
            // with the custom interpolation
            ConfigurationInterpolator.registerGlobalLookup("custom",
                    new CustomInterpolation());

            // Set the properties configuration.
            Configuration config = new CustomPropertiesConfiguration(
                    "ressources/macro.properties");

            String baseProp = "base.prop";
            String firstProp = "first.prop";
            String secondProp = "second.prop";

            ArrayList<Object> values = (ArrayList<Object>) config
                    .getList(baseProp);
            System.out.println(baseProp + "=>");
            for (int i = 0; i < values.size(); ++i) {
                System.out.println("[" + i + "]" + values.get(i));
            }

            System.out.println();

            values = (ArrayList<Object>) config.getList(firstProp);
            System.out.println(firstProp + "=>");
            for (int i = 0; i < values.size(); ++i) {
                System.out.println("[" + i + "]" + values.get(i));
            }

            System.out.println();

            values = (ArrayList<Object>) config.getList(secondProp);
            System.out.println(secondProp + "=>");
            for (int i = 0; i < values.size(); ++i) {
                System.out.println("[" + i + "]" + values.get(i));
            }

        } catch (ConfigurationException e) {
            e.printStackTrace();
        }

    }
}

For the tests I used the following properties file:

base.prop = /base, /root\\\\\\\\, t\\,t\\,t\\,t\\,
first.prop = ${custom:base.prop}, /first\\,/base
second.prop = ${custom:first.prop}, /second

And I get the following output:

base.prop=>
[0]/base
[1]/root\\
[2]t,t,t,t,

first.prop=>
[0]/base
[1]/root\\
[2]t,t,t,t,
[3]/first,/base

second.prop=>
[0]/base
[1]/root\\
[2]t,t,t,t,
[3]/first,/base
[4]/second

As you can see the solution is able to handle properties' values with backslash and the default delimiter ','. Some pattern containing this two elements might be not handle correctly, but this solution should handle basic values.