Blacklist URLs with headless Chrome

Question

I'm trying to block URLs in my specs, achieving something like I had when using capybara_webkit:

Capybara::Webkit.configure do |config|
  config.block_url("*google*")
  config.allow_url('*my_website.com')
end

After reading this article, I tried to do something like:

require 'webmock/rspec'

module WebmockConfig
  def self.default_disabled_urls
    [
      '*google*'
    ]
  end
end

WebMock.disable_net_connect!(allow_localhost: true)
WebMock.disable_net_connect!(allow: WebmockConfig.default_disabled_urls)

but I'm getting

Real HTTP connections are disabled. Unregistered request: POST http://127.0.0.1/session

even if that should be solved by WebMock.disable_net_connect!(allow_localhost: true).

When running the specs without WebMock.disable_net_connect!(allow: WebmockConfig.default_disabled_urls), everything is working fine.

Answer 1

The capybara-webkit white/blacklisting affects the requests made by the browser, whereas WebMock can only affect requests made by your app. This means WebMock is useless for what you want since it wouldn't actually stop your browser from loading anything from google, etc. To do that while using the selenium driver you need to use a programmable proxy like puffing-billy which will allow you to customize the responses for any matching requests the browser makes.

To configure a driver using headless chrome and puffing_billy you could do something like

Capybara.register_driver :headless_chrome do |app|
 browser_options = ::Selenium::WebDriver::Chrome::Options.new
 browser_options.headless!
 browser_options.add_argument("--proxy-server=#{Billy.proxy.host}:#{Billy.proxy.port}")
 Capybara::Selenium::Driver.new(app, browser: :chrome, options: browser_options)
end

Whether or not you need any other options is dependent on your system config, etc but you should be able to tell by looking at your current driver registration.

Answer 2

The allow_localhost: true settings are overwritten by allow: WebmockConfig.default_disabled_urls you have to call WebMock.disable_net_connect! once with both settings or by adding 'localhost', '127.0.0.1' entries into self.default_disabled_urls