Perform click on Web page element before parsing in Java

Question

I'm trying to parse HTML page with DOM parser and jsoup library. The problem that I'm facing is this:

On Web site there are two buttons which show two different tables. I need to parse the table which is shown when the second button is clicked. There are different attribute values set after clicking the second button.

When I do Jsoup.connect("example.com")

I get response like first button is selected and I don't need that data.

Is there a way to perform click on second button, and then start parsing and retrieving data from Web site?

Answer 1

Jsoup is just a parser, i.e. it can't handle events such as clicking on buttons. Have a look at browser automation tools (e.g. Selenium) to perform this kind of job.

Answer 2

JSoup can't control the web page, only parse the content. For manipulation and interaction, there are some tools. I recommend Geb, which uses a Groovy DSL with a JQuery like syntax, making it very fluent. It's also pretty easy to parse xml/html with it.