Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Watir timing out when trying to find any elements after visiting webpage

So probably the most important thing to preface this with is that I'm using c9. It's an IDE in the cloud and so that is giving me a lot of trouble when trying to use Chrome or Firefox with Watir, because I can't write a path to the Chrome or Firefox browser. I've also tried every variation of wait methods I could find but none of them work.

def save
    require 'watir'
    require 'phantomjs'

    @browser = Watir::Browser.new :phantomjs
    @browser.goto "https://kroger.softcoin.com/programs/kroger/digital_coupons/?origin=DigitalCoupons&banner=Smiths#contentBox"

    @browser.div(id: "contentBox").wait_until(&:present?).text
    @products = @browser.divs

end

Error

timed out after 30 seconds, waiting for true condition on #"contentBox", :tag_name=>"div"}>

The way I want to fix this problem of not being able to scrape data from the Smiths website is to use a chrome browser, but I get the error "unable to connect to chromedriver 127.0.0.1:9515"

like image 563
ChrisWilson Avatar asked Sep 16 '17 22:09

ChrisWilson


Video Answer


1 Answers

I had similar problem and I resolved it by installing docker container with selenium

# docker-compose.yml file
version: '2'
services:
  selenium:
    image: selenium/standalone-chrome
    ports:
      - "4444:4444"
    restart: always
    volumes:
      - "${PWD}/spec:${PWD}/spec" # I exposed `spec` dir to cover code with specs
      - /dev/shm:/dev/shm


# test.rb file
@browser = Watir::Browser.new(
  :remote,
  url: 'http://localhost:4444/wd/hub'
)

Run container with command:

docker run -it -d -P -p 4444:4444 -v `pwd`/spec:`pwd`/spec selenium/standalone-chrome

And try again

(Also you can run container even on VPS or another remote server, and then connect to it)

like image 181
itsnikolay Avatar answered Sep 30 '22 18:09

itsnikolay