I am trying to load the whole content of this link (clicking on المزيد)
I tried so many tutorials like this one here, which teaches how to work with a similar issue (Infinite Scrolling Pages).
My issue is that I couldn't manage to specify the load more class on this page to click it. But, if I am not mistaken, it exists in this part of the webpage source code:
<ng-include src="'loader.html'" class="ng-scope">
<div class="loading-div ng-scope ng-hide" ng-show="!loadingMoreData">
<div class="spinner">
<div class="bounce1"></div>
<div class="bounce2"></div>
<div class="bounce3"></div>
</div>
</div>
</ng-include>
<div class="block-hed block-link clearfix" data-ng-show="isMoreDisplayed" data-ng-click="getMoreMaterials()">
<h4 class="link-border-color"><a href="javascript:void(0)">المزيد </a></h4>
</div>
I do not necessarily need to implement any function like "click()" or "perform()". Any way to show the whole content under load more button is considered.
by the way, this is my code so far:
from selenium import webdriver
browser = webdriver.Chrome("/home/aziz/anaconda3/lib/python3.6/site-packages/chromedriver/chromedriver")
am = browser.get("https://sabq.org/%D8%A7%D9%84%D9%85%D9%85%D9%84%D9%83%D8%A9/%D8%A5%D9%82%D8%AA%D8%B5%D8%A7%D8%AF")
UPDATE I used this link here to solve the issue, but none of the tutorials worked.
find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector
I finally managed to solve the problem. I just added this line after my code
while True:
browser.find_elements_by_link_text('المزيد')[1].click()
and the page started infinitely loading all of the articles. I didn't really know that المزيد itself is clickable. I thought there is a link included in it.
The correct/better way to do it is to use WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
while True:
element = wait.until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, 'المزيد')))
element.click()
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With