Encoding::UndefinedConversionError ("\xE2" from ASCII-8BIT to UTF-8): error in ROR + MongoDB based app

Question

Had a developer write this method and its causing a Encoding::UndefinedConversionError ("\xE2" from ASCII-8BIT to UTF-8): error.

This error only happens randomly so the data going in is original DB field is what is causing the issue. But since I don't have any control over that, what can I put in the below method to fix this so bad data doesn't cause any issues?

def scrub_string(input, line_break = ' ')
  begin
     input.an_address.delete("^\u{0000}-\u{007F}").gsub("\n", line_break)
  rescue
     input || ''
  end
end

Will this work?

 input = input.encode('utf-8', :invalid => :replace, :undef => :replace, :replace => '_')

Answer 1

Yeah this should work, it'll replace any weird characters that can't be converted into UTF-8 with an underscore.

Read more about encoding strings in ruby here:

http://ruby-doc.org/core-1.9.3/String.html#method-i-encode