How do I start Thinking Sphinx delayed delta rake task from deploy script?

Question

I have Thinking Sphinx setup and working however I am having a problem getting the Delayed Job rake tasks to start during deployment.

I have the following task in deploy.rb which appears to execute, however the delayed jobs are not processed - they stack up until I run rake ts:dd from the server command line:

namespace :thinkingsphinx do
  desc 'Start Delayed Job Sphinx delta indexing'
  task :dd do
    run "cd #{current_path} && rake ts:dd RAILS_ENV=#{rails_env} &" 
  end
end

How can I get the delayed jobs to start running from the deployment script?

Thanks

Simon

Answer 1

The link Florian provided has code by Amit Solanki that works!

Here is what I did to get this to work with Capistrano:

Install gems

• ts-delayed-delta
• daemon_generator
• whenever

Create a file called script/delayed_delta with the contents:

#!/usr/bin/env ruby
require 'rubygems'
require 'daemons'
dir = File.expand_path(File.join(File.dirname(__FILE__), '..'))

daemon_options = {
  :multiple => false,
  :dir_mode => :normal,
  :dir => File.join(dir, 'tmp', 'pids'),
  :backtrace => true
}
puts 'delayed_delta'

Daemons.run_proc('job_runner', daemon_options) do
  if ARGV.include?('--')
    ARGV.slice! 0..ARGV.index('--')
  else
    ARGV.clear
  end

  Dir.chdir dir
  RAILS_ENV = ARGV.first || ENV['RAILS_ENV'] || 'development'
  require File.join('config', 'environment')

  Delayed::Worker.new(
    :min_priority => ENV['MIN_PRIORITY'],
    :max_priority => ENV['MAX_PRIORITY']
  ).start
end

Configure Capistrano

Capistrano needs to start Sphinx and job_runner (with our script/delayed_delta).

Add something like this to the deploy.rb:

deploy.task :restart, :roles => :app do
  run "export RAILS_ENV=production && cd #{deploy_to}/current && /usr/bin/rake ts:rebuild"  
  run "export RAILS_ENV=production && cd #{current_path} && /usr/bin/ruby script/delayed_delta start"
end

Configure whenever gem

In your config/schedule.rb add lines to update Sphinx's index and start job_runner if it isn't already running

every 30.minutes do
  command "export RAILS_ENV=production && cd /path/to/rails/production && /usr/bin/rake ts:index && /usr/bin/ruby script/delayed_delta start"
end

This gets converted to a crontab that is run every 30 minutes to update sphinx

Final Notes and Lessons Learned

• The script/delayed_delta uses the daemon_generator gem to start the job_runner background worker script. This is equivalent to running rake thinking_sphinx:delayed_deltas on the console, but persistent.

• Make sure only one job_runner or rake thinking_sphinx:delayed_deltas process is running at one time

• Let Capistrano start both Sphinx (rake ts:rebuild) and script/delayed_delta. I had problem when I started sphinx and delayed_deltas from different users or different environments

Answer 2

This link will be useful for anybody who need to restart the delayed_delta rake task from deploy script :

http://amitsolanki.com/2010/04/running-delayed-delta-daemon-in-background-for-thinking-sphinx/#comment-5802