Ruby 1.9 doesn't support Unicode normalization yet

Question

I'm trying to port over some of my old rails apps to Ruby 1.9 and I keep getting warnings about how "Ruby 1.9 doesn't support Unicode normalization yet." I've tracked it down to this function, but I'm getting about 20 warning messages per request:

rails-2.3.5/activesupport/lib/active_support/inflector.rb

def transliterate(string)
  warn "Ruby 1.9 doesn't support Unicode normalization yet"
  string.dup
end

Any ideas how I should start tracking these down and resolving it?

Answer 1

If you are aware of the consequences, i.e. accented characters will not be transliterated in Ruby 1.9.1 + Rails 2.3.x, place this in config/initializers to silence the warning:

# http://stackoverflow.com/questions/2135247/ruby-1-9-doesnt-support-unicode-normalization-yet
module ActiveSupport
  module Inflector
    # Calling String#parameterize prints a warning under Ruby 1.9,
    # even if the data in the string doesn't need transliterating.
    if Rails.version =~ /^2\.3/
      undef_method :transliterate
      def transliterate(string)
        string.dup
      end
    end
  end
end

Rails 3 does indeed solve this issue, so a more future-proof solution would be to migrate towards that.

Answer 2

The StringEx Gem seems to work pretty well. It has no dependency on Iconv either.

It adds some methods to the String class, like "to_ascii" which does beautiful transliteration out of the box:

require 'stringex'
"äöüÄÖÜßë".to_ascii #=> "aouAOUsse"

Also, the Babosa Gem does a great job transliterating UTF-8 strings, even with language support:

"Jürgen Müller".to_slug.transliterate.to_s           #=> "Jurgen Muller"
"Jürgen Müller".to_slug.transliterate(:german).to_s  #=> "Juergen Mueller"

Enjoy.