Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Sidekiq processing with local redis but not with remote

I have a RoR app with background jobs using whenever and sidekiq gems.

In development environment when I launch sidekiq with local redis instance (on localhost) the job keeps getting executed without problems. But when I switch to a remote redis instance (Heroku add-on) and restart sidekiq, it says it started processing, but nothing happens and workers aren't doing any jobs.

Here's my config/schedule.rb (for whenever gem)

every 2.minutes do
  rake "crawler:crawl"
end

Here's my initializers/redis.rb:

Sidekiq.configure_server do |config|
  config.redis = { :url => 'redis://user:[email protected]:9098/' }
end

Sidekiq.configure_client do |config|
  config.redis = { :url => 'redis://user:[email protected]:9098/' }
end

If I comment out the content in redis.rb and run a local redis instance, the jobs are processed normally. But when I use this remote redis instance, this shows up and then nothing gets processed:

2013-11-29T15:09:26Z 95156 TID-ov6y7e14o INFO: Booting Sidekiq 2.13.0 using redis://redistogo:[email protected]:9098/ with options {}
2013-11-29T15:09:26Z 95156 INFO: Running in ruby 1.9.3p327 (2012-11-10 revision 37606) [x86_64-darwin11.4.2]
2013-11-29T15:09:26Z 95156 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2013-11-29T15:09:26Z 95156 INFO: Starting processing, hit Ctrl-C to stop
like image 503
Matic Jurglič Avatar asked Nov 29 '13 15:11

Matic Jurglič


People also ask

How does Sidekiq work with Redis?

Sidekiq server process pulls jobs from the queue in Redis and processes them. Like your web processes, Sidekiq boots Rails so your jobs and workers have the full Rails API, including Active Record, available for use. The server will instantiate the worker and call perform with the given arguments.

Does Sidekiq require Redis?

Sidekiq is supported by Redis as a job management tool to process thousands of jobs in a second. Follow the steps to add Sidekiq and Redis to your existing application. Note: Redis gem is also installed in sidekiq, you don't have to install it independently.

Why does Sidekiq need Redis?

Sidekiq uses Redis as an in-memory data structure store and is written in Ruby. It also supports Java clients. It can be used with Resque, another Redis based job scheduler, or more commonly as a standalone product. Sidekiq reads jobs from a Redis queue, using the First In First Out (FIFO) model, to process jobs.

How do I run Sidekiq locally?

To run sidekiq, you will need to open a terminal, navigate to your application's directory, and start the sidekiq process, exactly as you would start a web server for the application itself. When the command executes you will see a message that sidekiq has started.


2 Answers

Maybe you connecting to wrong redis database or not connected at all. In my apps I use redis url without trailing slash. In your case:

This is for database "0"

redis://user:[email protected]:9098

And this for database "1"

redis://user:[email protected]:9098/1
like image 117
Andrey Artemyev Avatar answered Oct 06 '22 22:10

Andrey Artemyev


I use the environment variable REDIS_URL to ensure that everything is using the same Redis.

Re: Heroku - I just read this here as I was searching for my own solution:

If you're running on Heroku, you can't rely on the config/database.yml as that platform relies on the DATABASE_URL environment variable to determine the database connection configuration. Heroku overwrites the database.yml during slug compilation so that it reads from DATABASE_URL.

like image 29
Daniël W. Crompton Avatar answered Oct 06 '22 23:10

Daniël W. Crompton