Menu
I am trying to do some automated functionality using python selenium, and I'm coming across some weird behavior.
General layout of the html:
<html>
<body>
<div class="parent">
<iframe style="display: none"> ... </iframe>
<iframe style="display: none"> ... </iframe>
<iframe style="display: block">
#document
...
<div class="someClass"> ... </div>
</iframe>
<iframe style="display: none"> ... </iframe>
<iframe style="display: none"> ... </iframe>
</div>
</body>
Now, each iframe actually has the same inner html, and the code from the website seems to be randomly choosing which iframe is getting the display="block". However, I can't find any of the iframes.
I tried a standard way: iframe = driver.find_elements_by_xpath("//iframe[contains(@style, 'display:block')]")
That failing, I then tried just to find any iframe: driver.find_element_by_tag_name("iframe")
Neither of those found any iframe elements. I'm seeing the following error:
Traceback (most recent call last):
File "myfile.py", line 60, in <module>
iframe = driver.find_element_by_xpath("//iframe[contains(@style, 'display: block')]")
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: The result of the xpath expression "//iframe[contains(@style, 'display: block')]" is: [object HTMLIFrameElement]. It should be an element.
(Session info: chrome=93.0.4577.63)
Any thoughts on why the xpath is returning [object HTMLIFrameElement] and why I can't access that as I do other objects when searching by xpath?
Edit
New code option 1:
iframes = driver.find_elements_by_xpath(".//iframe[contains(@style,'display: block')]")
This still throws the exact same error as above
New code option 2:
parent = driver.find_element_by_xpath("//div[@class='parent']")
iframes = parent.find_elements_by_tag_name("iframe")
// when I print typeof iframes here, it's a list of dicts
// find the right index. Here, for simplicity, I just set it a default value
index = 4
// ...
driver.switch_to.frame(iframes[index])
I get the following error:
Traceback (most recent call last):
File "myfile.py", line 76, in <module>
driver.switch_to.frame(iframe)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\switch_to.py", line 89, in frame
self._driver.execute(Command.SWITCH_TO_FRAME, {'id': frame_reference})
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: missing 'ELEMENT'
(Session info: chrome=93.0.4577.82)
And when I print out iframes:
[{}, {}, {}, {}, {u'ontouchmove': {}, u'ontouchstart': {}}, {}, {}, {}, {}, {}]
For reference, this is the page I'm trying to hit. Sometimes you have to refresh a few times to get the challenge, and it comes far more frequent when using selenium. Also, using headless mode causes the challenge to happen everytime... https://catalog.usmint.gov/coins/coin-programs/morgan-and-peace-silver-dollar-coins/
681
The only situation I've seen like that was when trying to access an iframe from any ads of something similar, that was disappearing in the moment I was trying to get them.
I always solved that with the getElementsByTagName("iframe"), so try to wait a little more for the page to load before loading it, just to be sure the iframe was totally initialised before running it.
One way to do this was already discussed on this question
Also, here the official doc on the wait patterns for python: https://selenium-python.readthedocs.io/waits.html
PS: just tested your example html page and I could easily get them on my browser when using document.getElementsByTagName("iframe"), as you can see on the image below, so most probably you're running into one of the issues I've mentioned above, since your Selenium should be able to see them, assuming they are static and don't vanish and that your page fully loaded:
In your case, if you're receiving as attribute the HTMLIFrameElement instead of a simple iframe tag, it means you're dealing with a Web interface, from which you can access their attributes directly, and it means that indeed you found an iframe on the page.
You can use its properties to access the native APIs, and it has a .src property, reflecting the URL that's being loaded by it, and in many cases you could open this URL in a different page, and get what's rendered by it directly (unless the URL contains some CORS block).
Also, there's indeed some bugs on Selenium with Chrome related to WaitForPageToLoad and this can be fixed using other methods, as described here, though I don't think it's your current issue.
182
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
