how to get text between a specific span with HtmlUnit

Question

I'm new to HtmlUnit and I'm not even sure if it is the right tool for my project. I'm trying to parse a website and extract the values I need from it. I need to get the value "07:05" from this,

<span class="tim tim-dep">07:05</span>

I know that I can use the getTextContent() for extracting the value but I don't know how I can select a specific span. I used getElementById for finding the

<div>

tag that this expression belongs to but when I get the text content of that div, I get a whole line of text with a lot of unnecessary data. Can someone tell me how I can select this expression, possibly using the class name?

Answer 1

You need to browse a page and interact with it, like this:

final WebClient web = new HtmlUnit();
final HtmlPage page = web.getPage("http://www.whateveryouwant.com.br");

Get the elements by the tagname, and iterate over it:

final List<DomElement> spans = page.getElementTagName("span");
for (DomElement element : spans) {
    if (element.getAttribute("class").equals("tim tim-dep")) {
        return element.getNodeValue();
    }
}

Or just use XPath:

// Not sure what getFirstByXPath return
DomElement element = page.getFirstByXPath("//span[@class='tim tim-dep']");
final String text = element.getNodeValue();

Answer 2

here you go..

DomElement element = page.getFirstByXPath("//span[@class='tim tim-dep']");
String text = element.getTextContent();