Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Jsoup - extracting text

I need to extract text from a node like this:

<div>
    Some text <b>with tags</b> might go here.
    <p>Also there are paragraphs</p>
    More text can go without paragraphs<br/>
</div>

And I need to build:

Some text <b>with tags</b> might go here.
Also there are paragraphs
More text can go without paragraphs

Element.text returns just all content of the div. Element.ownText - everything that is not inside children elements. Both are wrong. Iterating through children ignores text nodes.

Is there are way to iterate contents of an element to receive text nodes as well. E.g.

  • Text node - Some text
  • Node <b> - with tags
  • Text node - might go here.
  • Node <p> - Also there are paragraphs
  • Text node - More text can go without paragraphs
  • Node <br> - <empty>
like image 247
Eugene Retunsky Avatar asked Apr 16 '12 16:04

Eugene Retunsky


3 Answers

Element.children() returns an Elements object - a list of Element objects. Looking at the parent class, Node, you'll see methods to give you access to arbitrary nodes, not just Elements, such as Node.childNodes().

public static void main(String[] args) throws IOException {
    String str = "<div>" +
            "    Some text <b>with tags</b> might go here." +
            "    <p>Also there are paragraphs</p>" +
            "    More text can go without paragraphs<br/>" +
            "</div>";

    Document doc = Jsoup.parse(str);
    Element div = doc.select("div").first();
    int i = 0;

    for (Node node : div.childNodes()) {
        i++;
        System.out.println(String.format("%d %s %s",
                i,
                node.getClass().getSimpleName(),
                node.toString()));
    }
}

Result:

1 TextNode 
 Some text 
2 Element <b>with tags</b>
3 TextNode  might go here. 
4 Element <p>Also there are paragraphs</p>
5 TextNode  More text can go without paragraphs
6 Element <br/>
like image 180
Vadim Ponomarev Avatar answered Nov 10 '22 13:11

Vadim Ponomarev


for (Element el : doc.select("body").select("*")) {

        for (TextNode node : el.textNodes()) {

                    node.text() ));

        }

    }
like image 20
Charles Avatar answered Nov 10 '22 14:11

Charles


Assuming you want text only (no tags) my solution is below.
Output is:
Some text with tags might go here. Also there are paragraphs. More text can go without paragraphs

public static void main(String[] args) throws IOException {
    String str = 
                "<div>"  
            +   "    Some text <b>with tags</b> might go here."
            +   "    <p>Also there are paragraphs.</p>"
            +   "    More text can go without paragraphs<br/>" 
            +   "</div>";

    Document doc = Jsoup.parse(str);
    Element div = doc.select("div").first();
    StringBuilder builder = new StringBuilder();
    stripTags(builder, div.childNodes());
    System.out.println("Text without tags: " + builder.toString());
}

/**
 * Strip tags from a List of type <code>Node</code>
 * @param builder StringBuilder : input and output
 * @param nodesList List of type <code>Node</code>
 */
public static void stripTags (StringBuilder builder, List<Node> nodesList) {

    for (Node node : nodesList) {
        String nodeName  = node.nodeName();

        if (nodeName.equalsIgnoreCase("#text")) {
            builder.append(node.toString());
        } else {
            // recurse
            stripTags(builder, node.childNodes());
        }
    }
}
like image 33
John Zoetebier Avatar answered Nov 10 '22 13:11

John Zoetebier