is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

Question

So im scraping links using selenium. I can print my links with my loop but I cant navigate to them because I get the following error:

selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

from selenium import webdriver

driver = webdriver.Firefox()

driver.get("https://www.famousgraphicdesigners.org/")

links = driver.find_elements_by_xpath('//*[@id="pages-2"]/div/ul/li/a')
links_total = len(links)
print("Found", links_total, "total links.\n")

for i in links:
    # print(i.get_attribute('href')) # This works
    driver.get(i.get_attribute('href')) # This doesnt work

driver.quit()

Answer 1

When you navigate to a new page the previously located elements become stale, in that case the elements in links, so you can't access the href attribute. Kee p all the hrefs in a list of strings and iterate over it

links = driver.find_elements_by_xpath('//*[@id="pages-2"]/div/ul/li/a')
links_hrefs = [link.get_attribute('href') for link in links]

for i in links_hrefs:
    driver.get(i)

Answer 2

Reason for this is because the element to which you have referred is removed from the DOM structure.

solution :

from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains

links = []
driver = webdriver.Chrome(executable_path=r"C:\New folder\chromedriver.exe")
driver.maximize_window()
driver.get('https://www.famousgraphicdesigners.org/')

all_Links = WebDriverWait(driver, 15).until(EC.presence_of_all_elements_located((By.XPATH, "//*[@id='pages-2']/div/ul/li/a[@href]")))

for link in all_Links:
    print link.get_attribute("href")
    links.append(link.get_attribute("href"))

for link in links:
    driver.get(link)