Grails not encoding the unicode characters properly

Question

In my grails app, the unicode characters are not being encoded properly.

I'm using grails 1.3.7 and tomcat 7.0.22. Following are the settings that I've configured in my app for unicode support:

- Set grails.views.gsp.encoding and grails.converters.encoding="UTF-8" in Config.groovy
- Set encoding to UTF-8 in meta tag in the .gsp pages
- Specified 'useUnicode=true&characterEncoding=UTF-8' to the MySql connection URL (DB has characterset set to UTF-8)
- Set URIEncoding="UTF-8" useBodyEncodingForURI="true" to the server.xml file in tomcat
- Specified the attribute accept-charset="UTF-8" of the form tag.

But still, when I submit a unicode character, grails doesn't support the character and the garbled value is being saved. I've googled around and have read ppl asking for help on this same issue but unfortunately the solutions don't work for my favor. Although, I've found a workaround to this problem. The following expression

params.detail = params.detail ? new String(params.detail.getBytes("8859_1"), "UTF8") : null

will correctly encode the unicode character.

However, using this approach is tedious since I will have to do this to all the text inputs in my app. Why is the unicode character not being properly encoded by grails and/or tomcat? I think I have the correct settings.

Answer 1

If you are not using Mysql but the HSqlDB which is shipped as default, you will not see your encoding problems. The problems occurs due to Mysql, InnoDB and UTF-8.

Since you are using a Mysql Connection and already setting useUnicode=true&characterEncoding=UTF-8 to the MySql connection URL

you still have to add a special hibernate dialect for InnoDB and UTF-8:

Datasource.groovy should contain:

environments {
    development {
        dataSource {         
            ......
            driverClassName = "com.mysql.jdbc.Driver"
            dialect = "com.domain.mysql.dialect.MySQLUTF8InnoDBDialect"
            .....

Create a new File in src/java/com/domain/mysql/dialect/MySQLUTF8InnoDBDialect.java

package com.domain.mysql.dialect;

import org.hibernate.dialect.MySQLInnoDBDialect;

/**
 * Sets the default charset to UTF-8.
 */
public class MySQLUTF8InnoDBDialect extends MySQLInnoDBDialect {

    @Override
    public String getTableTypeString() {
        return " ENGINE=InnoDB DEFAULT CHARSET=utf8";
    }
}

Make sure your Config.groovy has:

grails.views.default.codec = "html"
grails.views.gsp.encoding = "UTF-8"
grails.converters.encoding = "UTF-8"

And your views/layouts/main.gsp starts with:

<%@ page contentType="text/html;charset=UTF-8" %>

Greets,

Jan

Answer 2

Setting the Dialect to UTF-8 is not going to work. You have to also modify the tables or the columns of your table to support UTF-8.

Here is the command to alter the table

ALTER TABLE tbl_name CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;

or you can try to just modify the column in which you want to store utf-8

using the command

ALTER TABLE t MODIFY col1 CHAR(50) CHARACTER SET utf8;