RSpec: How to test methods that use Parallel (PG::ConnectionBad error)

Question

In my app I have several Builder classes that are responsible for taking data received from an external API request and building/saving resources to the database. I'm dealing with a large amount of data and have implemented the Parallel gem to speed this up by using multiple processes.

However, I'm finding that any test for a method that uses Parallel fails with the same error:

 ActiveRecord::StatementInvalid:
   PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

Here is an example of the code being tested:

class AirportBuilder < Resource
  def build_from_collection
    Parallel.each(object_producer, in_processes: 24) do |params|
      instance = Airport.find_or_initialize_by(fsid: params[:fs])
      build!(instance, params)
    end
  end
end

I've done some searching on this but all the results in Google have to do with using multiple threads/processes to make the test suite run faster, which is a different problem.

Any ideas on how I can test this effectively without causing the PG error? I realize I may need to stub something out but am not quite sure what to stub and still have a meaningful test.

Thanks in advance to anyone who might be able to help!

Answer 1

Are you using too many database connections than are configured for your test database? Maybe try setting it to a pool size equal to the needs of your script (which looks like 24)?

test: adapter: whatever host: whatever username: whatever password: whatever database: whatever pool: 24

Heads up that you may also want to do some math on the default ActiveRecord connection pool. Some good info in this Heroku dev center article.