Menu
I am new to Sidekiq and use it with Ruby on Amazon EC2 instances to do some work using ImageMagick with images.
While running it I realized that every worker runs on the same core. I use EC2 c3.2xlarge machines and they have 8 cores. It shows CPU usage is 15% but one core used 100%, and the others are using 0%.
Can Sidekiq use different CPU cores for different workers? If it can, is this inefficiency caused by ImageMagic and how can I make it to use other cores?
532
Sidekiq was the first multi-threading tool in the Ruby community.
By default Sidekiq uses 10 threads which means that either your database on your worker will need to be 10+ threads or you will need to configure your Sidekiq process to use fewer threads.
Concurrency - number of tasks that can be performed parallelly(threads) in sidekiq. If concurrency is 2 and 10 jobs come to sidekiq, it will execute 2 jobs at a time. Once the first two jobs are completed then only it can execute the next jobs.
If you want to utilize multiple cores using MRI, you'll need to start multiple Sidekiq processes; having multiple threads configured for your sidekiq instance is not enough.
So if you wanted to use all 8 cores, you would start 8 processes. They will all feed off of the same queue, so there's no need to worry about jobs being processed multiple times.
97
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
