Using Poltergeist, Phantom JS instances are not exiting during every rspec run

Question

On every poltergeist test that is executed by rspec, if I create a new session using:

Capybara.session_name="some_session_name"

a phantomjs instance is started as a subprocess, and never quits until the test ends, causing an OOM on my build server.

I believe this is due to a failure to call driver.quit, as described in the readme of Poltergeist:

If you run a few capybara sessions manually please make sure you've called session.driver.quit when you don't need session anymore. Forgetting about this causes memory leakage and your system's resources can be exhausted earlier than you may expect.

However, I call page.driver.quit in the after block of my tests.

Below is my after block code. $adhoc_sessions is a global variable I populate every time I set Capybara.session_name, with the value matching the value set on Capybara.session_name.

config.after(:each) do
  if example.metadata[:js]

    $adhoc_sessions.each do |session_name|
      Capybara.using_session( session_name ) do
      page.driver.quit
      end
    end
    $adhoc_sessions.clear
  end

Any suggestions on what I could do better here? Am I failing to call some cleanup command?

Answer 1

I found a solution that came from two constraints:

1. I don't think you can call driver.quit safely in Capybara without obtaining access to the private @session_pool, because Capybara has no way to let users remove a session from a pool once it gets put in. So, if you call driver.quit on a session, you can't remove that session from the pool, and eventually Capybara will try to reset! the session, causing Poltergeist to throw up an IOError because it's internal communication over websockets is not connected.
2. If you do instead whack the entire session pool after every test run, and quit every poltergeist driver within each session as you do so, eventually you'll run into a TOO MANY OPEN FILES error. I.e.,:

Method to recreate TOO MANY OPEN FILES error--do not use this!!

# you have to do quite a few test runs to cause the open files error
config.append_after(:each) do
  session_pool = Capybara.instance_variable_get("@session_pool")
  session_pool.each do | key, value |
    value.driver.quit
  end
  session_pool.clear
end    

I believe this to be a real poltergeist bug, but I don't care... and here's why... in running the above code, I noticed that creating a poltergeist session is a noticeably slow and resource intensive operation. So, I've decided I'd rather have an pool of sessions that never go away... the way Capybara appears to be designed.

The only problem with this approach becomes in using Capybara.session_name the way I do, which is to come up with arbitrary test names on a per test basis. Maybe in one test I want each session_name to be the same as a user's database ID. Or maybe I come up with 5 constants I use through out a test, and 5 different constants for a different test. In other words, I may use 100s of session_name's in my test suite, but I only ever have a maximum of just a handful sessions for anyone given test. So a good solution reuses poltergeist sessions, but let's me use arbitrary session name's per test run.

This is my solution

spec/utilities.rb

# holds a single test's session name's, mapped to pooled session names
$capybara_session_mapper = {}

# called after each test,
# to make sure each test run has it's own map of session names
def reset_session_mapper
  $capybara_session_mapper.clear
end

# manages the mapped session name
def mapped_session_name(session_name)
  return :default if session_name == :default # special treatment for the built-in session
  $capybara_session_mapper[session_name] ||= $capybara_session_mapper.length
end

# in place of ever using Capybara.session_name directly, 
# this utility is used to handle the mapping of session names in a way across all tests runs
def in_client(name)  
  Capybara.session_name = mapped_session_name(session_name)

  yield
end

In *spec_helper.rb*:

config.after(:each) do
  Capybara.reset_sessions!
  reset_session_mapper
end

An example test that uses in_client instead of Capybara.session_name directly:

it "can't see a private thing until it is made public" do

  in_client(user1.id) do
    visit '/some/private/thing'
    expect(page).to have_selector('#private-notice')
  end

  in_client(user2.id) do
    visit '/expose/some/private/thing'
  end

  in_client(user1.id) do
    visit '/some/private/thing`
    expect(page).to have_selector('#private-content')
  end
end

-- copied from my github answer