ActiveRecord unable to reconnect when running inside forked/threaded application?

Question

Bit of an open-ended question here, so I'll outline the problem first. We have a Resque worker who is supposed to pop data-synchronisation jobs off a queue, the reasons are two-fold, using Cron (and paying Rails environment boot time over-and-over) sucks, and the alternative queues, well Github made a pretty good case against them when they announced Resque. Also, the Redis time-series functionality already plays a big part of our infrastructure, where we also shuffle TS data into RRDTool & etc...

Here's the problem, with a typical three-hours between jobs (but staff can schedule jobs at any time… hence the queue), the PostgreSQL server goes away. Easy enough to cure, I expected that setting reconnect: true under the corresponding environment would ensure that this worked as expected.. I read in a few places that reconnect: true won't work for applications that use fork(). Which, naturally Resque does, to start it's workers… the part I don't understand is why ActiveRecord's reconnect can't work under these circumstances?

I noticed the implementations of reconnect! the MySQL Adapter and the PostgreSQL Adapter in ActiveRecord are different… but either way I would expect the ActiveRecord reconnect: true configuration to work.

The problem seems to be clear enough, when the child process exists, it closes the file handles created by the parent (thus hanging up the connection to the database) - is it possible to close a filehandle in such a way that ActiveRecord doesn't recognise that the connection has been terminated?

There's also, for what it's worth, an ActiveRecord aware fork() which I was able to find on Github, as a pastie - it's untested, but I assume it works (haven't tried it with current Rails..)

My question is more, Why can't the automatic reconnect in AR work if you fork()? (and, latterly - I can't be the only person who is having this problem; I'm chalking it up to using PGSQL with Resque!)

Answer 1

While not exactly the answer to "why can't it reconnect", but I think that what can help you is the following code put somewhere in the initialization phase:

Resque.after_fork do |job|
  ActiveRecord::Base.connection.reconnect!
end

Update: Regarding the reconnect - it seems that it's MySQL only feature. This is how it's used in mysql adapter: https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb#L848 As you can see, it uses the underlying driver's reconnect feature. https://github.com/kwatch/mysql-ruby/blob/master/ext/mysql.c#L923

On the other hand, the postgresql adapter doesn't do anything with regard to reconnect option, you can see the https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb

Also, the PostgreSQL site explicitly says that "Forking a process with open libpq connection can lead to unpredictable results" - http://www.postgresql.org/docs/9.0/interactive/libpq-connect.html And it's also evident that the C driver doesn't provide any reconnect feature.