Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

XPath to first occurrence of element with text length >= 200 characters

How do I get the first element that has an inner text (plain text, discarding other children) of 200 or more characters in length?

I'm trying to create an HTML parser like Embed.ly and I've set up a system of fallbacks where I first check for og:description, then I would search for this occurrence and only then for the description meta tag.

This is because most sites that even include meta description describe their site in that tag, instead of the contents of the current page.

Example:

<html>
    <body>
        <div>some characters
            <p>200 characters <span>some more stuff</span></p>
        </div>
    </body>
</html>

What selector could I use to get the 200 characters portion of that HTML fragment? I don't want the some more stuff either, I don't care what element it is (except for <script> or <style>), as long as it's the first plain text to contain at least 200 characters.

What should the XPath query look like?

like image 759
bevacqua Avatar asked Mar 06 '12 01:03

bevacqua


2 Answers

Use:

(//*[not(self::script or self::style)]/text()[string-length() > 200])[1]

Note: In case the document is an XHTML document (and that means all elements are in the xhrml namespace), the above expression should be specified as:

(//*[not(self::x:script or self::x:style)]/text()[string-length() > 200])[1]

where the prefix "x:" must be bound to the XHTML namespace -- "http://www.w3.org/1999/xhtml" (or as many XPath APIs call this -- the namespace must be "Registered" with this prefix)

like image 90
Dimitre Novatchev Avatar answered Nov 13 '22 09:11

Dimitre Novatchev


I meant something like this:

root.SelectNodes("html/body/.//*[(name() !='script') and (name()!='style')]/text()[string-length() > 200]")

Seems to work pretty well.

like image 26
bevacqua Avatar answered Nov 13 '22 09:11

bevacqua