How do you integrate Resque in Cucumber features?

Question

I've been trying to apply Square's method of including Resque in their integration tests without much luck. I'm not sure if Resque and/or Cucumber has changed a lot since August 2010.

Below you find the approach I took, and perhaps you can either:

1. Tell me where I went wrong, and how I can fix it
2. Recommend a completely new way of integrating Resque into Cucumber features

What I did to install it

Square's blog post didn't have explicit steps on how to install it, so this is what I did:

1. Downloaded their gist into features/support/cucumber_external_resque_worker.rb
2. Created a Rails initializer at config/initializers/cucumber_external_resque.rb that did the following:
   1. require 'features/support/cucumber_external_resque_worker'
   2. CucumberExternalResqueWorker.install_hooks_on_startup
3. In cucumber_external_resque_worker.rb, I changed instances of Rails.env.cucumber? to Rails.env.test? because Cucumber was running the features in the test environment (I did some puts Rails.env in cucumber_external_resque_worker.rb to be sure.
4. I run the features. At this point, I get stuck because I get the error uninitialized constant WorkerBase (NameError). Perhaps Resque has changed the way it names things.

Thanks in advance!

Answer 1

You can just run the Resque job synchronously by setting

Resque.inline = true

I added this line to my config/initializers/resque.rb:

Resque.inline = Rails.env.test?

It's more concise than the other approaches suggested in the post. As it is synchronous, it will be a little slower.

Answer 2

I spent a while toying with this today, and I think I have a solution.

Here's an updated Gist that removes the need for WorkerBase.

It also includes the config changes necessary to get things working (which are identical to the changes you discovered).

---

# This is adapted from this gist: https://gist.github.com/532100 by Square
# The main difference is that it doesn't require Bluth for WorkerBase
# It also calls prune_dead_workers on start so it doesn't hang on every other run
# It does not do anything special to avoid connecting to your main redis instance; you should be
# doing that elsewhere

class CucumberExternalResqueWorker
  DEFAULT_STARTUP_TIMEOUT = 1.minute
  COUNTER_KEY = "cucumber:counter"

  class << self
    attr_accessor :pid, :startup_timeout

    def start
      # Call from a Cucumber support file so it is run on startup
      return unless Rails.env.test?
      if self.pid = fork
        start_parent
        wait_for_worker_to_start
      else
        start_child
      end
    end

    def install_hooks_on_startup
      # Call from a Rails initializer
      return unless Rails.env.test?
      # Because otherwise crashed workers cause a fork and we pause the actual worker forever
      Resque::Worker.all.each { |worker| worker.prune_dead_workers }
      install_pause_on_start_hook
      install_worker_base_counter_patch
    end

    def process_all
      # Call from a Cucumber step
      unpause
      sleep 1 until done?
      pause
    end

    def incr
      Resque.redis.incr(COUNTER_KEY)
    end

    def decr
      Resque.redis.decr(COUNTER_KEY)
    end

    def reset_counter
      Resque.redis.set(COUNTER_KEY, 0)
    end

    private

    def done?
      Resque.redis.get(CucumberExternalResqueWorker::COUNTER_KEY).to_i.zero?
    end

    def pause(pid = self.pid)
      return unless Rails.env.test?
      Process.kill("USR2", pid)
    end

    def unpause
      return unless Rails.env.test?
      Process.kill("CONT", pid)
    end

    def start_parent
      at_exit do
        #reset_counter
        Process.kill("KILL", pid) if pid
      end
    end

    def start_child
      # Array form of exec() is required here, otherwise the worker is not a direct child process of cucumber.
      # If it's not the direct child process then the PID returned from fork() is wrong, which means we can't
      # communicate with the worker.
      exec('rake', 'resque:work', "QUEUE=*", "RAILS_ENV=test", "VVERBOSE=1")
    end

    def wait_for_worker_to_start
      self.startup_timeout ||= DEFAULT_STARTUP_TIMEOUT
      start = Time.now.to_i
      while (Time.now.to_i - start) < startup_timeout
        return if worker_started?
        sleep 1
      end

      raise "Timeout while waiting for the worker to start. Waited #{startup_timeout} seconds."
    end

    def worker_started?
      Resque.info[:workers].to_i > 0
    end

    def install_pause_on_start_hook
      Resque.before_first_fork do
        #reset_counter
        pause(Process.pid)
      end
    end

    def install_worker_base_counter_patch
      Resque.class_eval do
        class << self
          def enqueue_with_counters(*args, &block)
            CucumberExternalResqueWorker.incr
            enqueue_without_counters(*args, &block)
          end
          alias_method_chain :enqueue, :counters
        end
      end
      Resque::Job.class_eval do
        def perform_with_counters(*args, &block)
          perform_without_counters(*args, &block)
        ensure
          CucumberExternalResqueWorker.decr
        end
        alias_method_chain :perform, :counters
      end
    end
  end
end

In the Cucumber environment file features/support/env.rb

After:

require 'cucumber/rails'

Add:

require 'lib/cucumber_external_resque_worker'
CucumberExternalResqueWorker.start

Change:

  DatabaseCleaner.strategy = :transaction

to:

  DatabaseCleaner.strategy = :truncation

Also, create a file: config/initializers/cucumber_external_resque.rb

require 'lib/cucumber_external_resque_worker'
CucumberExternalResqueWorker.install_hooks_on_startup

# In my controller, I have:
  def start
    if params[:job] then
      Resque.enqueue(DemoJob, j.id)
    end
    redirect_to :action => :index
  end
# DemoJob:
class DemoJob
  def self.queue
    :demojob
  end
  def perform(job_id)
    j = Job.find(job_id)
    j.value = "done"
    j.save
  end

# In your actual Cucumber step file, for instance:
When /I click the start button for "([^"]*)"/ do |jobname|
  CucumberExternalResqueWorker.reset_counter
  Resque.remove_queue(DemoJob.queue)
  click_button(jobname)
end
When /all open jobs are processed/ do
  CucumberExternalResqueWorker.process_all
end

# And you cucumber feature file looks like:
# Scenario: Starting a job
#   When I click the start button for "Test Job 1"
#     And all open jobs are processed
#   Then I am on the job index page
#     And I should see an entry for "Test Job 1" with a value of "done".