How to block Referral Spam using Heroku

Question

I recently launched my first app on Heroku and I've been seeing traffic appear in Google Analytics to a page that doesn't exist on my app. I would not advise clicking on these sites but it is showing up under "top pages" as http://co.lumb.co/ and then showing http://forum.topic56809347.darodar.com/ as the "top referrer".

I've seen a few solutions that involve using the .htaccess file to block the spammer, but how would I do this with Heroku and Rails? (I'm new to both)

Is this something I should be concerned about? I found a similar question on this Wordpress thread: https://wordpress.org/support/topic/a-non-existent-page-is-showing-up-on-my-analytics

Thanks!

Answer 1

You can block requests at the Rack middleware layer. You can use a gem (rack-attack), or you can build your own Rack middleware.

Option 1: Rack-attack

https://github.com/kickstarter/rack-attack

If you want to go the rack-attack way, after you've installed it, you can use a config that looks like this:

# config/initializers/rack-attack.rb

class Rack::Attack
  blacklist("block referer spam") do |request|
    spammers = [/co\.lumb\.co/, /darodar/]
    spammers.find { |spammer| request.referer =~ spammer }
  end
end

Since we love testing, here's the test you should write first, before you implement the solution (using an RSpec request spec):

# spec/requests/referer_spam_block_spec.rb

require "rails_helper"

describe "Referer blacklist", type: :request do
  describe "referer spam" do
    it "is blocked" do
      spammers = ["http://co.lumb.co/", "http://forum.topic56809347.darodar.com/"]

      spammers.each do |spammer|
        get root_path, {}, { "HTTP_REFERER" => spammer }

        expect(response).to be_forbidden
      end
    end
  end

  describe "regular referer" do
    it "is not blocked" do
      get root_path, {}, { "HTTP_REFERER" => "google.com" }

      expect(response).to be_ok
    end
  end

  describe "direct request" do
    it "is not blocked" do
      get root_path

      expect(response).to be_ok
    end
  end
end

Option 2: Custom middleware

I haven't implemented this option (I used rack-attack), but here's a rough idea of what Rack middleware would look like to get this working (NOTE: I haven't tested this code out):

# lib/referer_spam_blocker.rb

class RefererSpamBlocker
  SPAMMERS = [/co\.lumb\.co/, /darodar/]

  def initialize(app)
    @app = app
  end

  def call(env)
    if blacklisted?(env)
      forbidden
    else
      @app.call(env)
    end
  end

  private

  def blacklisted?(env)
    !SPAMMERS.find { |spammer| env["HTTP_REFERER"] =~ spammer }.empty?
  end

  def forbidden
    [403, {'Content-Type' => 'text/plain'}, ["Forbidden\n"]]
  end
end

and then you can configure Rails to use this by adding this to config/application.rb

config.middleware.use RefererSpamBlocker

More details on how to write Rack middleware here: http://railscasts.com/episodes/151-rack-middleware

Bonus

If you want to be able to change what you're blocking without changing your code, you can use ENV vars instead of hardcoding the spammers in.