Menu
I'm trying to iterate through the text content of a subtree using elt.itertext() (v3.5.0b1) as follows:
import lxml.html.soupparser as soupparser
import requests
doc = requests.get("http://f10.5post.com/forums/showthread.php?t=1142017").content
tree = soupparser.fromstring(doc)
nodes = tree.getchildren()
for elt in nodes:
for t in elt.itertext():
print t
But I keep getting an error saying
File "src/lxml/iterparse.pxi", line 248, in lxml.etree.iterwalk.__init__ (src/lxml/lxml.etree.c:134032)
File "src/lxml/apihelpers.pxi", line 67, in lxml.etree._rootNodeOrRaise (src/lxml/lxml.etree.c:15220)
ValueError: Input object has no element: HtmlComment
Is there a way to skip all HTML comments? Also, what does this error actually mean?
Thanks
484
This is normal.
>>> from lxml import etree
>>> doc = '''
... <html><!-- PAGENAV POPUP -->
... <div class="vbmenu_popup" id="pagenav_menu" style="display:none">
... <table cellpadding="4" cellspacing="1" border="0">
... <tr>
... <td class="thead" nowrap="nowrap">Go to Page...</td>
... </tr>
... <tr>
... <td class="vbmenu_option" title="nohilite">
... <form action="index.php" method="get" onsubmit="return this.gotopage()" id="pagenav_form">
... <input type="text" class="bginput" id="pagenav_itxt" style="font-size:11px" size="4" />
... <input type="button" class="button" id="pagenav_ibtn" value="Go" />
... </form>
... </td>
... </tr>
... </table>
... </div>
... <!-- / PAGENAV POPUP -->
... </html>'''
>>> root = etree.fromstring(doc)
>>> nodes = root.getchildren()
>>> nodes
[<!-- PAGENAV POPUP -->, <Element div at 0x10367f290>, <!-- / PAGENAV POPUP -->]
>>> for elt in nodes:
... for t in elt.itertext():
... print t
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "lxml.etree.pyx", line 1406, in lxml.etree._Element.itertext (src/lxml/lxml.etree.c:48845)
File "lxml.etree.pyx", line 2763, in lxml.etree.ElementTextIterator.__cinit__ (src/lxml/lxml.etree.c:64747)
File "iterparse.pxi", line 219, in lxml.etree.iterwalk.__init__ (src/lxml/lxml.etree.c:125303)
File "apihelpers.pxi", line 72, in lxml.etree._rootNodeOrRaise (src/lxml/lxml.etree.c:13689)
ValueError: Input object has no element: lxml.etree._Comment
As you can see above
>>> nodes
[<!-- PAGENAV POPUP -->, <Element div at 0x10367f290>, <!-- / PAGENAV POPUP -->]
Note: getchildren is deprecated. You can use list.
>>> list(root)
[<!-- PAGENAV POPUP -->, <Element div at 0x10367f290>, <!-- / PAGENAV POPUP -->]
The nodes is a list of Element and comments. If you check how itertext() is working:
Creates a text iterator. The iterator loops over this element and all subelements, in document order, and returns all inner text.
On the other hand if instead of iterating on the list, I was iterating directly on the root element with:
>>> for t in root.itertext():
... print t
...
I get all the text and a lot of spaces. :)
if you still want to iterate on a list of nodes. You can infer the nature with
>>> [item.tag for item in nodes]
[<built-in function Comment>, 'div', <built-in function Comment>]
You can also do
>>> [item.__class__ for item in nodes]
[<type 'lxml.etree._Comment'>, <type 'lxml.etree._Element'>, <type 'lxml.etree._Comment'>]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
