How can I make this code to submit a UTF-8 form textarea with jQuery/Ajax work?

Question

I am having problems submitting forms which contain UTF-8 strings with Ajax. I am developing a Struts web application which runs in a Tomcat server. This is the environment I set up to work with UTF-8:

• I have added the attributes URIEncoding="UTF-8" useBodyEncodingForURI="true" into the Connector tag to Tomcat's conf/server.xml file.

• I have a utf-8_general_ci database

• I am using the next filter to ensure my request and responses are encoded in UTF-8

  package filters;
  
  import java.io.IOException;
  import javax.servlet.*;
  
  public class UTF8Filter implements Filter {
      public void destroy() {}
  
      public void doFilter(ServletRequest request,ServletResponse response, FilterChain chain)
          throws IOException, ServletException {
          request.setCharacterEncoding("UTF-8");
          response.setContentType("text/html;charset=UTF-8");
          chain.doFilter(request, response);
      }
  
      public void init(FilterConfig filterConfig) throws ServletException {
      }
  }
  

• I use this filter in WEB-INF/web.xml

• I am using the next code for my JSON responses:

  public static void populateWithJSON(HttpServletResponse response,JSONObject json)
  {
     String CONTENT_TYPE="text/x-json;charset=UTF-8";
     response.setContentType(CONTENT_TYPE);
     response.setHeader("Cache-Control", "no-cache");
     try {
          response.getWriter().write(json.toString());
     } catch (IOException e) {
      throw new ApplicationException("Application Exception raised in RetrievedStories", e);
     }
  }
  

Everything seems to work fine (content coming from the database is displayed properly, and I am able to submit forms which are stored in UTF-8 in the database). The problem is that I am not able to submit forms with Ajax. I use jQuery, and I thought the problem was the lack of contentType field in the Ajax request. But I was wrong. I have a really simple form to submit comments which contains of an id and a body. The body field can be in different languages such as Spanish, German, or whatever.

If I submit my form with body textarea containing contraseña, Firebug shows me:

Request Headers

• Host localhost:8080
• Accept-Charset ISO-8859-1, utf-8;q=0.7;*q=0.7
• Content-Type application/x-www-form-urlencoded; charset UTF-8

If I execute Copy Location with parameters in Firebug, the encoding seems already wrong:

http://localhost:8080/Cerepedia/corporate/postStoryComment.do?&body=contrase%C3%B1a&id=88

This is my jQuery code:

function addComment() {
    var comment_body = $("#postCommentForm textarea").val();
    var item_id = $("#postCommentForm input:hidden").val();
    var url = rooturl+"corporate/postStoryComment.do?";
    $.post(url, { id:  item_id, body: comment_body } ,
        function(data){
        /* Do stuff with the answer */
    }, "json");  }

A submission of a form with jQuery is causing the next error server side (note I am using Hibernate).

javax.servlet.ServletException: org.hibernate.exception.GenericJDBCException: Could not execute JDBC batch update
    at org.apache.struts.action.RequestProcessor.processException(RequestProcessor.java:520)
    at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:427)
    at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
    at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
    at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:462)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at com.cerebra.cerepedia.security.AuthorizationFilter.doFilter(AuthorizationFilter.java:78)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at com.cerebra.cerepedia.hibernate.HibernateSessionRequestFilter.doFilter(HibernateSessionRequestFilter.java:30)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at filters.UTF8Filter.doFilter(UTF8Filter.java:14)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:261)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:581)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
    at java.lang.Thread.run(Unknown Source)
Caused by: org.hibernate.exception.GenericJDBCException: Could not execute JDBC batch update
    at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:103)
    at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:91)
    at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
    at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:249)
    at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:235)
    at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:139)
    at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
    at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
    at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
    at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:338)
    at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)
    at com.cerebra.cerepedia.item.dao.ItemDAOHibernate.addComment(ItemDAOHibernate.java:505)
    at com.cerebra.cerepedia.item.ItemManagerPOJOImpl.addComment(ItemManagerPOJOImpl.java:164)
    at com.cerebra.cerepedia.struts.item.ItemAction.addComment(ItemAction.java:126)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:269)
    at org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:170)
    at org.apache.struts.actions.MappingDispatchAction.execute(MappingDispatchAction.java:166)
    at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
    ... 26 more
Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xF1a' for column 'body' at row 1
    at com.mysql.jdbc.ServerPreparedStatement.executeBatch(ServerPreparedStatement.java:657)
    at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)
    at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:48)
    at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:242)
    ... 44 more
26-ago-2008 19:54:48 org.apache.catalina.core.StandardWrapperValve invoke
GRAVE: Servlet.service() para servlet action lanzó excepción
java.sql.BatchUpdateException: Incorrect string value: '\xF1a' for column 'body' at row 1
    at com.mysql.jdbc.ServerPreparedStatement.executeBatch(ServerPreparedStatement.java:657)
    at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)
    at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:48)
    at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:242)
    at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:235)
    at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:139)
    at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
    at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
    at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
    at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:338)
    at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)
    at com.cerebra.cerepedia.item.dao.ItemDAOHibernate.addComment(ItemDAOHibernate.java:505)
    at com.cerebra.cerepedia.item.ItemManagerPOJOImpl.addComment(ItemManagerPOJOImpl.java:164)
    at com.cerebra.cerepedia.struts.item.ItemAction.addComment(ItemAction.java:126)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:269)
    at org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:170)
    at org.apache.struts.actions.MappingDispatchAction.execute(MappingDispatchAction.java:166)
    at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
    at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
    at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
    at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:462)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at com.cerebra.cerepedia.security.AuthorizationFilter.doFilter(AuthorizationFilter.java:78)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at com.cerebra.cerepedia.hibernate.HibernateSessionRequestFilter.doFilter(HibernateSessionRequestFilter.java:30)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at filters.UTF8Filter.doFilter(UTF8Filter.java:14)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:261)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:581)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
    at java.lang.Thread.run(Unknown Source)
javax.servlet.ServletException: java.lang.NumberFormatException: null
    at org.apache.struts.action.RequestProcessor.processException(RequestProcessor.java:520)
    at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:427)
    at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
    at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
    at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at com.cerebra.cerepedia.security.AuthorizationFilter.doFilter(AuthorizationFilter.java:78)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at com.cerebra.cerepedia.hibernate.HibernateSessionRequestFilter.doFilter(HibernateSessionRequestFilter.java:30)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at filters.UTF8Filter.doFilter(UTF8Filter.java:14)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:261)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:581)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NumberFormatException: null
    at java.lang.Long.parseLong(Unknown Source)
    at java.lang.Long.valueOf(Unknown Source)
    at com.cerebra.cerepedia.struts.item.ItemAction.addComment(ItemAction.java:120)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:269)
    at org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:170)
    at org.apache.struts.actions.MappingDispatchAction.execute(MappingDispatchAction.java:166)
    at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
    ... 26 more
26-ago-2008 20:13:25 org.apache.catalina.core.StandardWrapperValve invoke
GRAVE: Servlet.service() para servlet action lanzó excepción
java.lang.NumberFormatException: null
    at java.lang.Long.parseLong(Unknown Source)
    at java.lang.Long.valueOf(Unknown Source)
    at com.cerebra.cerepedia.struts.item.ItemAction.addComment(ItemAction.java:120)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:269)
    at org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:170)
    at org.apache.struts.actions.MappingDispatchAction.execute(MappingDispatchAction.java:166)
    at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
    at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
    at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
    at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at com.cerebra.cerepedia.security.AuthorizationFilter.doFilter(AuthorizationFilter.java:78)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at com.cerebra.cerepedia.hibernate.HibernateSessionRequestFilter.doFilter(HibernateSessionRequestFilter.java:30)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at filters.UTF8Filter.doFilter(UTF8Filter.java:14)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:261)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:581)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
    at java.lang.Thread.run(Unknown Source)

Answer 1

have you tried adding the following before the call :

$.ajaxSetup({ 
    scriptCharset: "utf-8" , 
    contentType: "application/json; charset=utf-8"
});

The options are explained here.

contentType : When sending data to the server, use this content-type. Default is "application/x-www-form-urlencoded", which is fine for most cases.

scriptCharset : Only for requests with 'jsonp' or 'script' dataType and GET type. Forces the request to be interpreted as a certain charset. Only needed for charset differences between the remote and local content.

Answer 2

I'm having the same problem. I saw Internet Explorer 8 sends this header:

content-type = application/x-www-form-urlencoded

while Firefox sends this:

content-type = application/x-www-form-urlencoded; charset=UTF-8

My solution was just forcing in jQuery to use the Firefox content-type:

$.ajaxSetup({ scriptCharset: "utf-8" ,contentType: "application/x-www-form-urlencoded; charset=UTF-8" });