EOFError when running feature tests with Capybara and headless Chrome [closed]

Question

EDIT: It turns out this whole problem was due to a hidden and poorly named monkey patch in our project for WEBrick, which Capybara uses as its default web server. StackOverflow isn't letting me delete the question, so be wary when reading further.

We're attempting to upgrade our Ruby version from 2.2.7 to 2.4.3. The app itself is running fine, but all of our feature tests that run via Capybara and headless Chrome (with Chromedriver) are suddenly failing.

Here's an example of the exact error we're seeing:

Failures:

  1) New UI - Settings Page - API with an unpermitted tier clicking on the "Request API Key" button shows a feature popup
     Got 0 failures and 2 other errors:

     1.1) Failure/Error: Unable to find matching line from backtrace

          EOFError:
            end of file reached

     1.2) Failure/Error: Unable to find matching line from backtrace

          EOFError:
            end of file reached

And here's how we're configuring the Selenium driver for headless Chrome:

CHROME_BINARY = ENV.fetch('CHROME_BINARY', '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome')

Capybara.register_driver :headless_chrome do |app|
  Capybara::Selenium::Driver.new(
    app,
    browser: :chrome,
    desired_capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(
      'chromeOptions' => {
        'binary' => CHROME_BINARY,
        'args' => %w(no-sandbox disable-gpu window-size=1440,900 headless)
      }
    )
  )
end

Ruby version: 2.4.3

Chromedriver version: 2.35.528157

Capybara version: 2.18.0

RSpec version: 3.7.0

selenium-webdriver: 3.9.0

Chrome: 64.0.3282.167

All other research I've done indicates that updating Chromedriver to the latest version will solve this issue, but we're already using the latest version (2.3.5).

I haven't been able to find a way to get any more of a backtrace for the above error. The spec isn't even getting to the point where it's loading anything from the database, it just fails immediately.

Another thing to note: this same exact behavior is occurring both on our CI server and on our local development machines. We use CircleCI with a Dockerized Linux container, and MacOS Sierra/High Sierra for our development machines.

We are not using chromedriver-helper or any other gem which installs chromedriver automatically. Installation of chromedriver was done either by brew install chromedriver or by downloading the latest chromedriver from the chromedriver website and placing in our $PATH.

Answer 1

UPDATE: the source of this problem was a monkeypatch to WEBrick. It was poorly named, and located in an unusual location and the author is no longer on the team. A good reminder that monkeypatches should be avoided at all costs.

---

Original answer:

I’ve been working with Jordan (OP) on the issue. While we haven’t found the root cause, we have found where the EOFError with no backtrace came from.

The following code can be found in Ruby’s standard lib: Net::BufferedIO#rbuf_fill

https://github.com/ruby/ruby/blob/v2_4_3/lib/net/protocol.rb#L185

# callers do not care about backtrace, so avoid allocating for it
raise EOFError, 'end of file reached', []

My guess is that the author of this code expected that this error would be rescued at some point. Perhaps there is a bug that allowed the error to reach us, and without a backtrace.

As I said, we never found the root cause. We did however narrow it down to the point that we were able to find an acceptable work around. The issue appears to be related to Webrick which is the server that capybara will use by default if it is not specified otherwise

• When a new capybara session is initialized, an instance of Capybara::Server is ‘booted’ this was webrick in our case: https://github.com/teamcapybara/capybara/blob/2.18_stable/lib/capybara/session.rb#L88
• Part of the boot process is to check if the server is “responsive”: https://github.com/teamcapybara/capybara/blob/2.18_stable/lib/capybara/server.rb#L105
• The responsive method makes a get request to the server (127.0.0.1): https://github.com/teamcapybara/capybara/blob/2.18_stable/lib/capybara/server.rb#L82
• At this point the rest is in ruby’s http lib. Eventually it fails with the EOFError here: https://github.com/ruby/ruby/blob/v2_4_3/lib/net/protocol.rb#L185

I assume that Webrick responded abnormally, or did not respond at all. If anyone has further insight on the matter, I’d love to learn.

In our case the workaround was simple. The app was already using puma in other environments, so we decided to try puma with capybara to see if it’d get us around this problem:

Capybara.server = :puma

Because Capybara’s rack server is no longer used, we’re no longer affected.

If anyone else runs into the same problem or has ideas regarding the root cause, please share!

Edit 1:

While stepping through code execution, I grabbed the result of caller when I reached rbuf_fill. For those that are interested, I've uploaded it here: https://gist.github.com/benjaminwood/c7c0d39fcfb2efd8a9085874cac07c36

Edit 2:

Updated info regarding Webrick as Capybara's default server. Thanks @Thomas Walpole for clarifying that.

Edit 3:

Add note regarding resolution (it was a monkeypatch).