Google app engine datastore string encoding problem

Question

Hello I'm using Google App Engine for a project I'm doing and I need to store some Strings. I'm using Java and JDOHelper.getPersistenceManagerFactory("transactions-optional")

While debugging it on my computer everything works fine and the Strings are saved correctly. But when i upload it to google app engine, all the Strings i save will have their unicode characters replaced by question marks (?). If I go to the DataViewer on the project page, I can see that the Strings are actually saved with question marks.

Like I said, when running it on my computer it works fine. Is there anyone who knows what I should do?

Answer 1

It sounds like you were not specifying the encoding for your HTTP POST content. Have a look at this question for details.

Answer 2

Like Jackrabbit said, you should specify charset. I still had some troubles on Google App Engine. After setting charset to UTF-8 AND using Spring's CharacterEncodingFilter nothing has bothered me encoding wise.

See How To Get Character Encoding Correct, which includes information on this code to add to your web.xml file:

<filter>
    <filter-name>SetCharacterEncoding</filter-name>
    <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
    <init-param>
        <param-name>encoding</param-name>
        <param-value>UTF-8</param-value>
    </init-param>
    <init-param>
       <param-name>forceEncoding</param-name>
       <param-value>true</param-value>
    </init-param>
</filter>
 <filter-mapping>
    <filter-name>SetCharacterEncoding</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

Be sure to add this as the first step in your filter chain!

Also, the author of the blog suggests setting the charsets in your JSP pages to utf-8 as well:

<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>