Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is server and client terminology mean in Sidekiq?

In SideKiq the documention says

Sidekiq.configure_server do |config|
  config.redis = {:namespace => "figs_#{Rails.env}", :size => 25, :url => 'redis://localhost:6379/0'}
end
Sidekiq.configure_client do |config|
  config.redis = {:namespace => "figs_#{Rails.env}", :size => 25, :url => 'redis://localhost:6379/0'}
end

I am curious about what does this configure_server and configure_client mean here ?

     config.redis = {:namespace => "figs_#{Rails.env}", :size => 25, :url => 'redis://localhost:6379/0'}

Is obviously the location of redis, queue type etc.

like image 541
Sahil Grover Avatar asked Feb 10 '14 05:02

Sahil Grover


People also ask

What is concurrency in Sidekiq?

Sidekiq's "concurrency" setting is just a setting of how many threads it will run at once. High numbers lead to more fragmentation. You could imagine that 1 thread uses log(x) memory over time, so increasing the number of threads leads to n * log(x) memory use.

How do I start a Sidekiq server?

To run sidekiq, you will need to open a terminal, navigate to your application's directory, and start the sidekiq process, exactly as you would start a web server for the application itself. When the command executes you will see a message that sidekiq has started.

What is a Sidekiq process?

Sidekiq is usually the background job processor of choice for Ruby-on-Rails, and uses Redis as a data store for the job queues. Background (or asynchronous) job processing is critical to GitLab because there are many tasks that: Shouldn't tie up relatively expensive HTTP workers to perform long-running operations.

How does Sidekiq queue work?

It works the same way as the scheduled queue; it contains jobs that have to be executed at a given time in the future. The main difference is that in retries queue, jobs are added by the Sidekiq, and the time when the job should be executed is calculated based on the retries counter.


2 Answers

The client is anything that pushes jobs to Redis, typically your passenger, puma or unicorn process running Rails or Sinatra. The server is the Sidekiq process which pulls jobs from Redis. One complexity: a Sidekiq server process can push new jobs to Redis thus acting like a client too!

config/sidekiq.yml is meant to allow the same config as command line args. The initializer is meant for more complicated config which requires Ruby, for instance the Redis connection info or custom middleware.

like image 83
Mike Perham Avatar answered Nov 05 '22 07:11

Mike Perham


I like to think about it as it applies to Heroku. There are two main processes (dynos) web, and worker.

Dynos that are of type web are where your application (web server) runs from and thus all connections from the router are delivered to your web dynos for your application to serve a response to.

Dynos that are of type worker are where your background jobs are processed on. It is where Sidekiq executes it's perform method in, that way it doesn't block on your web dyno.

It is in web dynos that Sidekiq.configure_client is run from. It configures how your web dyno connects to Sidekiq and submits jobs.

It is in worker dynos that Sidekiq.configure_server is run from. It configures how your worker dyno connects to Sidekiq and processes jobs.

like image 27
Joseph Viscomi Avatar answered Nov 05 '22 06:11

Joseph Viscomi