Heroku: unable to connect to chromedriver 127.0.0.1:9515 when using Watir/Selenium

Question

This runs locally (without specifying driver_path), but not on Heroku.

Code:

Selenium::WebDriver::Chrome.driver_path = ENV['GOOGLE_CHROME_SHIM']
browser = Watir::Browser.new :chrome

I've confirmed below values in heroku rails console

ENV['GOOGLE_CHROME_BIN'] => "/app/.apt/opt/google/chrome/chrome"
ENV['GOOGLE_CHROME_SHIM'] => "/app/.apt/usr/bin/google-chrome-stable"

Installed Buildpacks:

https://github.com/heroku/heroku-buildpack-chromedriver
https://github.com/heroku/heroku-buildpack-google-chrome

Current Error:

Selenium::WebDriver::Error::WebDriverError: unable to connect to chromedriver 127.0.0.1:9515

Searching unable to connect to chromedriver 127.0.0.1:9515 on SO returns a bunch of results but none mention heroku.

Also:

I considered phantomjs. Someone else got it working here, Using a headless browser with Heroku Rails Unicorn stack

But its deprecated. Below error when running it locally.

Selenium support for PhantomJS is deprecated. Use headless Chrome/Firefox or HTMLUnit instead.

Also Tried:

For transparency, I also tried the following.

Changing browser = Watir::Browser.new :chrome

To browser = Watir::Browser.new :chrome, headless: true

Though I didn't expect this to work.

Also Also Tried:

Removing: https://github.com/heroku/heroku-buildpack-chromedriver

Adding: https://github.com/heroku/heroku-buildpack-xvfb-google-chrome.

Adding headless gem.

And running the below script given in the watir gem page, http://watir.com/guides/headless/.

require 'watir'
require 'headless'
headless = Headless.new
headless.start
b = Watir::Browser.start 'www.google.com'
puts b.title
b.close
headless.destroy

Error:

Selenium::WebDriver::Error::UnknownError: unknown error: cannot find Chrome binary

I assume this failed because I didn't specify location of the chrome binary/shim. Couldn't find how to specify this when using headless in the docs.

Tried Per Suggestion:

heroku run /usr/bin/chromedriver --app app-name

Running /usr/bin/chromedriver on ⬢ app-name... up, run.2151

(Hobby) bash: /usr/bin/chromedriver: No such file or directory

Also see below logs that mention chrome when deploying to heroku:

remote: -----> chromedriver app detected
remote: -----> Looking up latest chromedriver version...
remote: -----> Downloading chromedriver v2.33...
remote: Archive:  /tmp/chromedriver.zip
remote:   inflating: /tmp/build_cd35072c5b766edaa2b565cbff57e5d6/.chromedriver/bin/chromedriver  
remote: -----> Creating chromedriver export scripts...
remote: -----> Google Chrome app detected
...
remote: -----> Fetching https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
remote: -----> Installing google-chrome-stable_current_amd64.deb
...
remote: -----> Creating google-chrome shims

Some Progress:

If I ssh into the heroku server,

heroku run bash --app app-name

And search for files with the name chrome,

find /app/ -name "*chrome*"

/app/.profile.d/chromedriver.sh
/app/.profile.d/010_google-chrome.sh
/app/.apt/etc/cron.daily/google-chrome
/app/.apt/opt/google/chrome
/app/.apt/opt/google/chrome/chrome
/app/.apt/opt/google/chrome/chrome_100_percent.pak
/app/.apt/opt/google/chrome/chrome-sandbox
/app/.apt/opt/google/chrome/chrome_200_percent.pak
/app/.apt/opt/google/chrome/google-chrome
/app/.apt/opt/google/chrome/cron/google-chrome
/app/.apt/usr/bin/google-chrome-stable
/app/.apt/usr/bin/google-chrome
/app/.apt/usr/share/menu/google-chrome.menu
/app/.apt/usr/share/doc/google-chrome-stable
/app/.apt/usr/share/applications/google-chrome.desktop
/app/.apt/usr/share/gnome-control-center/default-apps/google-chrome.xml
/app/.apt/usr/share/man/man1/google-chrome.1
/app/.apt/usr/share/appdata/google-chrome.appdata.xml
/app/vendor/bundle/ruby/2.4.0/gems/selenium-webdriver-3.7.0/lib/selenium/webdriver/chrome
/app/vendor/bundle/ruby/2.4.0/gems/selenium-webdriver-3.7.0/lib/selenium/webdriver/chrome.rb
/app/vendor/bundle/ruby/2.4.0/gems/browser-2.4.0/test/unit/chrome_test.rb
/app/vendor/bundle/ruby/2.4.0/gems/browser-2.4.0/lib/browser/platform/chrome_os.rb
/app/vendor/bundle/ruby/2.4.0/gems/browser-2.4.0/lib/browser/chrome.rb
/app/.chromedriver
/app/.chromedriver/bin/chromedriver

I can see chromedriver binary at /app/.chromedriver/bin/chromedriver.

So I tried

heroku run /app/.chromedriver/bin/chromedriver --app app-name

Result:

Running /app/.chromedriver/bin/chromedriver on ⬢ app-name... up, run.2067 (Hobby)
Starting ChromeDriver 2.33.506092 (733a02544d189eeb751fe0d7ddca79a0ee28cce4) on port 9515
Only local connections are allowed.

But then running heroku run rake selenium_namespace:task_one --app app-name gives the same result.

Selenium::WebDriver::Error::WebDriverError: unable to connect to chromedriver 127.0.0.1:9515 ... /app/vendor/ruby-2.4.1/lib/ruby/2.4.0/net/http.rb:906:in `rescue in block in connect': Failed to open TCP connection to 127.0.0.1:9515 (Connection refused - connect(2) for "127.0.0.1" port 9515) (Errno::ECONNREFUSED) ...

Answer 1

This is possible on Heroku.

---

Confusing chrome and chromedriver

Your configuration is mixing up chromedriver and Chrome. GOOGLE_CHROME_SHIM points to the Chrome executable google-chrome-stable, not to chromedriver. The line below results in Selenium executing the wrong binary, which results in the misleading error message.

Selenium::WebDriver::Chrome.driver_path = ENV['GOOGLE_CHROME_SHIM'] # WRONG!

As of writing this (Jan '18) the chromedriver build pack automatically adds /app/.chromedriver/bin to the $PATH variable. If you delete the above line Selenium should again be able to find chromedriver automatically.

And then?

You probably added the line above to fix Selenium not being able to find the Chrome binary. The error message for that would have looked something like:

Selenium::WebDriver::Error::UnknownError: unknown error: cannot find Chrome binary

You can fix this by telling Selenium where the Chrome binary is located using Selenium::WebDriver::Chrome::Options. The following code should accomplish that.

options = Selenium::WebDriver::Chrome::Options.new
chrome_bin_path = ENV.fetch('GOOGLE_CHROME_SHIM', nil)
options.binary = chrome_bin_path if chrome_bin_path # only use custom path on heroku
options.add_argument('--headless') # this may be optional
driver = Selenium::WebDriver.for :chrome, options: options
driver.navigate.to "https://stackoverflow.com"

Buildpacks

This should all be possible with the standard chrome and chromedriver build packs:

https://github.com/heroku/heroku-buildpack-google-chrome.git https://github.com/heroku/heroku-buildpack-chromedriver.git

You may need heroku-buildpack-xvfb-google-chrome instead of vanilla chrome if you're automating clicks in the browser, but that should not be required just to get headless chrome running.