The main menu of this page (linio) has 11 links. Only interested in 9 (those with gray background and show submenus when hovered).
I want to click every single element in the submenu from the 9 options. The desired process is:
1.-First section: "Celulares y Tablets".
2.-Go to: "Celulares y Smartphones". Do click and see this page.
3.-Extract some data (checked, I've been able to do this).
4.-Go to the next submenu in "Celulares y Tablets". Which is: "Accesorios Celular".
5.-Extract some data, and go to the next submenu. After done with all the submenus in this section, I would go to the next big section: "TV-Audio-y-Foto".
And so on with the 9 sections.
HTML Estructure
Looking the source code, I've arrived to this:
1.- Main Header: the main header is within a 'nav' tag:
<nav id="headerMainMenu>
2.- Inside the 'nav' tag is a 'ul', and every 'il' inside has and 'id' for each one of the 9 sections:
<nav id="headerMainMenu>
<ul>
<il id = "category-item-celulares-y-tablets"><a href="..."></il>
<il id = "category-item-celulares-y-tablets"><a href="..."></il>
<il id = "category-item-celulares-y-tablets"><a href="..."></il>
</ul>
</nav>
3.- Inside the il elements, there are div elements containing the links we need: Please, notice the <a>
with the class ="subnav__title".
<nav id="headerMainMenu>
<ul>
<il id = "category-item-celulares-y-tablets"><a href="...">
<div class="col-3">
<a href="..."class="subnav__title">TV y Video</a>
</il>
<il id = "category-item-celulares-y-tablets"><a href="..."></il>
<il id = "category-item-celulares-y-tablets"><a href="..."></il>
</ul>
</nav>
4.- Using RSelenium to go to each section:
library(RSelenium)
library(rvest)
#start RSelenium
checkForServer()
startServer()
remDr <- remoteDriver()
remDr$open()
#navigate to your page
remDr$navigate("http://www.linio.com.pe/")
#Accesing the first submenu from "Category Celulares y Tablets
webElem <- remDr$findElement(using = 'css', value = "#category-item-celulares-y-tablets a.subnav__title")
webElem$sendKeysToElement(list(key = "enter"))
But doing so shows this error:
> webElem$sendKeysToElement(list(key = "enter"))
Error: Summary: StaleElementReference
Detail: An element command failed because the referenced element is no longer attached to the DOM.
class: org.openqa.selenium.StaleElementReferenceException
*I think this question could be of help. But I don't get it.
**I think my CSS is Okay.
We can click on an element which is hidden with Selenium webdriver. The hidden elements are the ones which are present in the DOM but not visible on the page. Mostly the hidden elements are defined by the CSS property style="display:none;".
Selenium by default cannot handle hidden elements and throws ElementNotVisibleException while working with them. Javascript Executor is used to handle hidden elements on the page. Selenium runs the Javascript commands with the executeScript method. The commands to be run are passed as arguments to the method.
Solution: You need to scroll to element using javascript or Actions class so that element is perfectly visible on screen. By using explicit wait and fluent wait as mentioned above.
I used the following code for Python. I'm sure it can be converted to your language:
def click_hidden(self, css_selector):
'''
Click on a hidden element using javascript.
Selenium will error if the element doesn't excist and if javascript fails
REASON: Selenium doesn't allow clicks on hidden elements since the user won't either
So be sure the element would be visible in normal uses!
'''
element = self.find_css(css_selector)
self.execute_script("$(arguments[0]).click();", element)
return element
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With