Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to scroll to the end of the page using selenium in Python?

Tags:

I am trying to scroll to the end of a page so that I can make all the data visible and extract it. I tried to find a command for it but it's available in java (driver.executeScript) but couldn't find for python. Right now I am making the computer press the end key thousand times:

while i<1000:     scroll = driver.find_element_by_tag_name('body').send_keys(Keys.END)     i+=1 

And I also tried driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") but it scrolls to the end of the loaded page and the same thing END key does. Once at the bottom of the page, next content loads. But now it doesn't scroll again.

I know there will be a very nice alternative for this.

How do I scroll to the end of the page using selenium in Python?

like image 955
psr Avatar asked Sep 04 '15 06:09

psr


2 Answers

Well I finally figured out a solution:

lenOfPage = driver.execute_script("window.scrollTo(0, document.body.scrollHeight);var lenOfPage=document.body.scrollHeight;return lenOfPage;")     match=False         while(match==False):                 lastCount = lenOfPage                 time.sleep(3)                 lenOfPage = driver.execute_script("window.scrollTo(0, document.body.scrollHeight);var lenOfPage=document.body.scrollHeight;return lenOfPage;")                 if lastCount==lenOfPage:                     match=True 
like image 159
psr Avatar answered Sep 23 '22 18:09

psr


This can be done in one line by scrolling to document.body.scrollHeight

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") 
like image 45
AlexG Avatar answered Sep 26 '22 18:09

AlexG