Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

mysterious rails error with almost no trace

We're having a strange problem with one crawler. Occasionally it will throw a Rails FATAL error on some request, but the trace is very limited and looks something like this

[2014-07-01 18:16:37] FATAL Rails :
ArgumentError (invalid %-encoding (c ^   FK+ 9u$_    t  Kl
ΥE!   =k \  ̕* ߚ>c+<O   یo ʘ> C     R! 2 D  (5      x q#!` 4 p      |8 I   E
:+   H^9`^ #    Vo{   >

  =[z     )):
  lib/locale_middleware.rb:14:in `call'

The crawler user-agent is

Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html)

We can ask it to stop crawling us via robots.txt, but it would be better to deal with the root cause and not fail with 500 on those requests if possible.

We can't really reproduce this kind of request either, so any suggestions on how to generate a similar request would be of great help.

We're using Rails 3.2.19, Unicorn on Ubuntu 12.04. Here's our locale_middleware.rb

like image 425
gingerlime Avatar asked Jul 07 '14 13:07

gingerlime


1 Answers

Special thanks to Benjamin Sinclaire for pointing to the right issue on github.

The solution was described on this comment:

  • Install the rack-robustness gem
  • add this to application.rb:
config.middleware.use ::Rack::Robustness do |g|
  g.no_catch_all
  g.on(ArgumentError) { |ex| 400 }
  g.content_type 'text/plain'
  g.body{ |ex| ex.message }
  g.ensure(true) { |ex| env['rack.errors'].write(ex.message) }
end
like image 194
gingerlime Avatar answered Sep 28 '22 17:09

gingerlime