Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Trouble getting few items from a webpage

Tags:

I've written a script in python in combination with selenium to parse some items from a webpage. I can't get it working in anyway. The items I'm after are (perhaps) within iframe. I tried to switch it but that doesn't have any effect. I'm still getting nothing except for TimeoutException when it hits the line where I tried to switch the iframe. How can i get it working. Thanks in advance:

Here goes the webpage link: URL

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

url = "replace_with_above_url"

driver = webdriver.Chrome()
driver.get(url)
wait = WebDriverWait(driver, 10)

wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "tradingview_fe623")))

for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".quick .apply-common-tooltip"))):
    print(item.text)

driver.quit()

Elements within which the items I'm after:

<div class="quick">
    <span class="apply-common-tooltip">5</span>
    <span class="apply-common-tooltip">1h</span>
    <span class="apply-common-tooltip selected">1D</span>
    <span class="apply-common-tooltip">1M</span>
    <span class="apply-common-tooltip">1D</span>
</div>

This is the output I'm expecting to have (it works locally when i try to get them using css selectors):

5
1h
1D
1M
1D

This is how it looks in the web:

enter image description here

like image 585
SIM Avatar asked Dec 18 '17 18:12

SIM


1 Answers

Required nodes located inside 2 nested iframes, so you need to switch to them one by one. Note that id/name of second one generated dynamically. Just try to replace

wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "tradingview_fe623")))

with

wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, ".abs")))
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[id^='tradingview_']")))
like image 148
Andersson Avatar answered Sep 19 '22 13:09

Andersson