Menu
I am trying to write a program in Python that click to the next page until the it reaches to the last page. I followed some old posts on Stackoverflow and wrote the following code:
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
driver = webdriver.Chrome(executable_path="/Users/yasirmuhammad/Downloads/chromedriver")
driver.get("https://stackoverflow.com/users/37181/alex-gaynor?tab=tags")
while True:
try:
driver.find_element_by_link_text('next').click()
except NoSuchElementException:
break
However, when I run the program, it throws following error:
selenium.common.exceptions.WebDriverException: Message: unknown error: Element <a href="/users/37181/alex-gaynor?tab=tags&sort=votes&page=3" rel="next" title="go to page 3">...</a> is not clickable at point (1180, 566). Other element would receive the click: <html class="">...</html>
(Session info: chrome=68.0.3440.106)
I also followed a thread of Stackoverflow (selenium exception: Element is not clickable at point) but no luck.
952
You need to close this banner first -
Since selenium opens a fresh browser instance so the website will ask you to store cookies every time you run the script. It is this exact banner which is coming in the way of selenium clicking your "next" button. Use this code to delete that close button -
driver.find_element_by_xpath("//a[@class='grid--cell fc-white js-notice-close']").click()
Also, driver.find_element_by_link_text('next') will throw a StaleElementReferenceException. Use this locator instead -
driver.find_element_by_xpath("//span[contains(text(),'next')]").click()
Final code -
driver.get("https://stackoverflow.com/users/37181/alex-gaynor?tab=tags")
driver.find_element_by_xpath("//a[@class='grid--cell fc-white js-notice-close']").click()
while True:
try:
time.sleep(3)
driver.find_element_by_xpath("//span[contains(text(),'next')]").click()
except NoSuchElementException:
break
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With