What's up with those requests having "iframe=true&width=80%&height=80%" query params?

Question

I'm running a Rails 3.2 App. I checked Google Webmaster tools and saw lot's of HTTP 502 errors for random pages. Weird thing is that all of them where crawled with ?iframe=true&width=80%&height=80% as query param:

e.g. http://www.mypage.com/anypage?iframe=true&width=80%&height=80%

For sure I dont link like that to those pages internally, must be external. Checking Google, proofs me here - I see lot's of other pages having same issues.

Seems like an external service creates those links, but why??

Answer 1

I'm seeing these too. Over the past 24 hours I have 9 hits on one of my pages. They all come from the same IP address, which is Google's in Mountain View. None of them have a referrer. Also, a really interesting thing is that half of them have headers like this:

HTTP_ACCEPT           : */*
HTTP_ACCEPT_ENCODING  : gzip,deflate
HTTP_CONNECTION       : Keep-alive
HTTP_FROM             : googlebot(at)googlebot.com
HTTP_HOST             : mydomain.com
HTTP_USER_AGENT       : Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

But then interspersed are requests from the same IP that don't have any HTTP headers reported in the exception. I'm not sure if this means they aren't being sent, or if something in the Rails stack is preventing the headers from getting recorded due to some other variation in the requests. In any case the requests are interspersed.

The page in question has existed for only about a month, and it's only seen 5 requests during that time according to GA.

All this leads me to believe that someone inside Google is doing something experimental which is leading to these buggy query string encodings, and Rails apps are seeing it because it happens to crash the rack QS parser, whereas other platforms may be more forgiving.

In the meantime I may monkey patch rack just to stop shouting at me, but the ultimate answer about what's going on will have to come from Google (anyone there?).

Answer 2

You can add this to your initializers to get rid of the errors (with Ruby 1.8.x):

module URI

  major, minor, patch = RUBY_VERSION.split('.').map { |v| v.to_i }

  if major == 1 && minor < 9
    def self.decode_www_form_component(str, enc=nil)
      if TBLDECWWWCOMP_.empty?
        tbl = {}
        256.times do |i|
          h, l = i>>4, i&15
          tbl['%%%X%X' % [h, l]] = i.chr
          tbl['%%%x%X' % [h, l]] = i.chr
          tbl['%%%X%x' % [h, l]] = i.chr
          tbl['%%%x%x' % [h, l]] = i.chr
        end
        tbl['+'] = ' '
        begin
          TBLDECWWWCOMP_.replace(tbl)
          TBLDECWWWCOMP_.freeze
        rescue
        end
      end
      str = str.gsub(/%(?![0-9a-fA-F]{2})/, "%25")
      str.gsub(/\+|%[0-9a-fA-F]{2}/) {|m| TBLDECWWWCOMP_[m]}
    end
  end

end

All this does is encode % symbols that aren't followed by two characters instead of raising an exception. Not sure it's such a good idea to be monkeypatching rack, though. There must be a valid reason this wasn't done in the gem (maybe security related?).