Concurrency using Sidekiq is causing some problems

Question

I am using Sidekiq in my rails app to queue up 50k+ jobs at a time. Our pool size is set to 9.

The jobs are all related and do the same thing. We have another model that has a counter on it. During each job, we check to see if that model has a column with value above 200. If it is above 200, we create another instance of that model with value = 0 and continue the jobs. However, since we have 9 jobs running at a time, all 9 jobs read the value of that column to be greater than 200 at the same time and all create new instances, which isn't right.

What's the best way to solve for this issue? We basically want all jobs to read from the most up-to-date value.

Answer 1

I can't post any specific code because it will depend heavily on your database type and settings, but you should try database locking.

Worker when reading table should lock it until it finishes with creating new record with value 0. You should lock table for read so other workers will need to wait until this one worker finish. It is also possible to lock separate rows, but I don't know if it will work in your case.