Background job taking twice the time that the same operation within rails

Question

In my Rails application, I have a long calculation requiring a lot of database access.

To make it short, my calculation took 25 seconds.

When implementing the same calculation within a background job (a big single worker), the same calculation take twice the same time (ie 50 seconds). I have try several technics to put the job in a background process put none add an impact on my performances => using DelayJob / Sidekiq / doing the process within my rails but in a thread created for the work, but all have the same impact on my performances *2.

This performance difference only exist in rails 'production' environment. It looks like there is an optimisation done by rails that is not done in my background job.

My technical environment is the following =>

• I am using ruby 2.0 / rails 4
• I am using unicorn (but I have same problem without it).
• The job is using Rails.cache to store some partial computation.
• I am using postgresql

Does anybody has an clue where this impact might come from ?

Answer 1

I'm assuming you're comparing the background job speed to the speed of running the operation during a web request? If so, you're likely benefiting from Rails's QueryCache, which caches db queries during a web request. Try disabling it like described here:

Disabling Rails SQL query caching globally

If that causes the web request version of the job to take as long as the background job, you've found your culprit. You can then enable the query cache on your background job to speed it up (if it makes sense for your application).