UTF-8 encoding and http parameters

Question

I am doing a simple ajax call with the YahooUI Javascript library as follows:

YAHOO.util.Connect.setForm('myform');
YAHOO.util.Connect.asyncRequest('POST', url, ...);

Following are the settings in my app: Tomcat version: 6.0.18

Tomcat server connector : URIEncoding="UTF-8" webapp page :

Also stated in YahooUI connector library docs:

setForm will encode each HTML form field's name and value using encodeURIComponent. This results in a string of UTF-8 encoded, name-value pairs. NOTE: Setting an HTTP header of "Content-Type" with a different charset value will not change the encoding of the serialized data.encoding of the serialized data.

I can see that the french characters that are being sent as parameters are encoded (in ie7 using iehttpheader tool):

    name=%C3%88%C3%A0%C3%B4 
    testParam=%C3%B4%C3%B4   

For the data : name: Èàô and testParam: ôô

But on the server side I am seeing values as following: ÈàÃÂ

Even if I am converting the string to bytes and then create new string with the charset defined as follows: String val = new String(oo.getBytes("UTF-8")); I am not able to get the exact data as expected.

Note: I have referenced the question below, but was not able to resolve this issue: How to get UTF-8 working in Java webapps? Please guide.

UPDATE: Calling the decode UTF-8 function as given on W3.org site http://www.w3.org/International/O-URL-code.html is giving me the expected results. I expect Tomcat to be decoding this?.

Answer 1

Since you are using a POST query, URIEncoding="UTF-8" is not applicable here. You need to set a filter to tell Tomcat that your request encoding is UTF-8. You may use, for example, a Spring's CharacterEncodingFilter (usage, javadoc). Other implementations of such filters can also be found.

Answer 2

Yes, here is one:

package com.lfantastico.web;

import java.io.IOException;
import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;

public class EncodingFilter implements Filter {
    private String encoding = "UTF-8";

    public void destroy() {
    }

    public void doFilter(ServletRequest request, ServletResponse response,
            FilterChain chain) throws IOException, ServletException {
        request.setCharacterEncoding(encoding);
        response.setCharacterEncoding(encoding);
        chain.doFilter(request, response);
    }

    public void init(FilterConfig config) throws ServletException {
        if (config.getInitParameter("encoding") != null) {
            encoding = config.getInitParameter("encoding");
        }
    }
}