Reading unicode from messageSource gives problem with Java 5

Question

I made an Spring(2.5.6) webapplication with i18n support with property files (ex: messages_en_US.properties, messages_de_DE.properties).

This .properties files with uni-codes. for example:

busy = Besch\u00E4ftigt

When reading busy keyword from the messageSource gives this result:

...
private static ReloadableResourceBundleMessageSource messageSource;

    /**
     * Gets a message from the resources (.properties) defined in the applicationContext.xml
     *
     * @param input string to hook up
     * @return the the message hooked up from the resources
     */
    public static String getMessage(String input){
        System.out.println(input); //busy
        System.out.println(messageSource.getDefaultEncoding()); //UTF-8
        System.out.println(messageSource.getMessage(input, null, null)); //Beschu00E4ftigt
        return messageSource.getMessage(input, null, null);
    }
...

so without the \

The files on the server are also UTF-8:

The environments where the problem occurred:

• Tomcat 5.5.28 (Run jsp-api.jar and servlet-api.jar from common/lib )
• JDK 1.5.0_22

• JSTL 1.1.2 (read from application lib)

• Tomcat 6.0.32 (Run jsp-api.jar and servlet-api.jar from lib )

• JDK 1.5.0_22
• JSTL 1.1.2 (read from application lib)

The environments where the problem is solved (exactly the same distribution): - Tomcat 6.0.32 (Run jsp-api.jar and servlet-api.jar from lib ) - JDK 1.6.0_13 - JSTL 1.1.2 (read from application lib)

Please let me know if you need more information. And don't say I need to update my JDK because this isn't possible.

Update binding messageSource in applicationContext.xml

<b:bean id="messageSource" class="org.springframework.context.support.ReloadableResourceBundleMessageSource">
    <b:property name="defaultEncoding" value="UTF-8"/>
    <b:property name="fallbackToSystemLocale" value="false" />
    <b:property name="basenames">
        <b:list>
            <b:value>classpath:messages</b:value>
            <b:value>/public/custom/i18n/portalmessages</b:value>
        </b:list>
    </b:property>    
    <b:property name="cacheSeconds" value="1"/>
</b:bean>

Update 2: Place resource property file on classpath and with classloader:

URLClassLoader cl = (URLClassLoader) IOUtils.class.getClassLoader();
InputStream resourceAsStream = cl.getResourceAsStream("messages_de_DE.properties");
Properties prop = new Properties();
prop.load(resourceAsStream);
System.out.println("From classpath --> " + prop.get("busy")); //Beschäftigt
System.out.println("From i18n folder --> " + I18nFunctions.getMessage("busy")); //Beschu00E4ftigt

Answer 1

I had a look at the Source code of DefaultPropertiesPersister (it's used by ReloadableResourceBundleMessageSource internally).

If a defaultEncoding is specified, the properties are loaded manually line-by-line from a Reader instead of using the conventional Properties.load() method.

Before adding the key/value pair to the Properties object, the unescape() method is invoked on the Strings

protected String unescape(String str) {
    StringBuffer outBuffer = new StringBuffer(str.length());
    for (int index = 0; index < str.length();) {
        char c = str.charAt(index++);
        if (c == '\\') {
            c = str.charAt(index++);
            if (c == 't') {
                c = '\t';
            }
            else if (c == 'r') {
                c = '\r';
            }
            else if (c == 'n') {
                c = '\n';
            }
            else if (c == 'f') {
                c = '\f';
            }
        }
        outBuffer.append(c);
    }
    return outBuffer.toString();
}

This is where the \ character is getting removed.

If you create a subclass of DefaultPropertiesPersister as follows

package com.something;

import org.apache.commons.lang.StringEscapeUtils;
import org.springframework.util.DefaultPropertiesPersister;

public class MyPropertiesPersister extends DefaultPropertiesPersister {
    protected String unescape(String str)
    {
        return StringEscapeUtils.unescapeJava(str);
    }    
}

Set it in your spring config like so:

<b:bean id="messageSource" class="org.springframework.context.support.ReloadableResourceBundleMessageSource">
    <b:property name="defaultEncoding" value="UTF-8"/>
    <b:property name="fallbackToSystemLocale" value="false" />
    <b:property name="basenames">
        <b:list>
            <b:value>classpath:messages</b:value>
            <b:value>/public/custom/i18n/portalmessages</b:value>
        </b:list>
    </b:property>    
    <b:property name="cacheSeconds" value="1"/>
    <b:property name="propertiesPersister">
        <b:bean class="com.something.MyPropertiesPersister"/>
    </b:property>
</b:bean>

It will work.. there may be further jiggery-pokery required to get exactly what you want in relation to other encodings, etc :)