Python Selenium how to wait before clicking on link

Question

I just wonder, how to have the browser wait before clicking on a link? My goal is that I'm scraping from a dynamic webpage, the content is dynamic but I manage to get the form id. The only problem is that the submit button is only displayed after 2-3 seconds. However, my Firefox driver start clicking on the link immediately when the page is loaded (not the dynamic part).

Is there any way that I can make my browser wait 2-3 seconds until the submit button appears? I tried to use time.sleep() but it pauses everything, the submit button doesn't appear during time.sleep but appears after 2-3 seconds when time.sleep ends.

Answer 1

You need to use Selenium Waits.

In particular, element_to_be_clickable expected condition is what fits better than others:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

element = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.ID, "myDynamicElement"))
)
element.click()

where driver is your webdriver instance, 10 is the amount of seconds to wait for an element. With this setup, selenium would try to locate an element every 500 milliseconds for 10 seconds in total. It would throw TimeoutException after 10 seconds left if element would not be found.

Answer 2

You can set wait like following :

Explicit wait :

    element = WebDriverWait(driver, 20).until(
    EC.presence_of_element_located((By.ID, "myElement"))

Implicit wait :

 driver.implicitly_wait(20) # seconds
 driver.get("Your-URL")
 myElement = driver.find_element_by_id("myElement")

You can use any of above. Both are valid.