Unicode problem with JSF and HTML forms?

Question

I have an HTML form generated by JSF which maps an input element to a bean setter and it looks to me like JSF is garbling unicode input on the way in. In particular I put the following exception for testing purposes in the setter

public void setTitle(String title){
    System.out.println("title set with: "+title+"\n");
    if (title.startsWith("xxx")) {
        throw new RuntimeException("debug exception "+title);
    }
    this.title = title;
}

Then I put the following text into the form title input element: "xxxx 海陆". Then when I submit the form I see the log print

title set with: xxxx ????? 

(on a unicode compatible mac terminal). And I get an error message on the response HTML page:

Error setting property 'title' in bean of type   
uk.ac.lancs.e_science.sakaiproject.api.blogger.post.Post: 
java.lang.RuntimeException: debug exception xxxx ���??

Any clues on what's wrong? Am I just full of it and have the wrong diagnosis? I think I've eliminated all other possibilities. Unicode seems to work fine in other components of the same application.

Answer 1

Questions I would be asking:

• How is the form encoding the request (application/x-www-form-urlencoded or multipart/form-data)? Multi-part data will be decoded using a 3rd party MIME parser, so there is scope for trouble there. If the data is url-encoded, is it being escaped properly?
• What charsets is the browser accepting?
• What encoding is the server detecting? Is it a Unicode character set?
• Is it just the logging that is writing as a lossy encoding (e.g. MacRoman)? What default charset is the server using?

Since what you see on a console isn't necessarily what is in the string, you can dump the Unicode code points using this code:

  public static void printCodepoints(char[] s) {
    for (int i = 0; i < s.length; i++) {
      int codePoint = Character.isHighSurrogate(s[i]) ? Character
          .toCodePoint(s[i], s[++i])
          : s[i];
      System.out.println(Integer.toHexString(codePoint));
    }
  }