So probably the most important thing to preface this with is that I'm using c9. It's an IDE in the cloud and so that is giving me a lot of trouble when trying to use Chrome or Firefox with Watir, because I can't write a path to the Chrome or Firefox browser. I've also tried every variation of wait methods I could find but none of them work.
def save
require 'watir'
require 'phantomjs'
@browser = Watir::Browser.new :phantomjs
@browser.goto "https://kroger.softcoin.com/programs/kroger/digital_coupons/?origin=DigitalCoupons&banner=Smiths#contentBox"
@browser.div(id: "contentBox").wait_until(&:present?).text
@products = @browser.divs
end
Error
timed out after 30 seconds, waiting for true condition on #"contentBox", :tag_name=>"div"}>
The way I want to fix this problem of not being able to scrape data from the Smiths website is to use a chrome browser, but I get the error "unable to connect to chromedriver 127.0.0.1:9515"
I had similar problem and I resolved it by installing docker container with selenium
# docker-compose.yml file
version: '2'
services:
selenium:
image: selenium/standalone-chrome
ports:
- "4444:4444"
restart: always
volumes:
- "${PWD}/spec:${PWD}/spec" # I exposed `spec` dir to cover code with specs
- /dev/shm:/dev/shm
# test.rb file
@browser = Watir::Browser.new(
:remote,
url: 'http://localhost:4444/wd/hub'
)
Run container with command:
docker run -it -d -P -p 4444:4444 -v `pwd`/spec:`pwd`/spec selenium/standalone-chrome
And try again
(Also you can run container even on VPS or another remote server, and then connect to it)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With