I'd like to scrape a website that dynamically generates more content as I scroll down the web browser. I have seen a related post, Auto-Scroll in FireFox, but it doesn't answer my question.
Is it possible to scroll a webpage until the end of page (no more content is generated by the web server) or until a few refreshes using Watir web-driver?
I have recently tried to do something like that, and to my surprise looks like webdriver does not have support for scrolling. I did find two workarounds.
This will send space
to the browser, and it will scroll down (works on twitter.com, for example):
browser.send_keys :space
This will scroll to the element, and if the element is at the bottom of the page, it will load more content:
browser.element.wd.location_once_scrolled_into_view
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With