How to code an automated bot that can browse and do operations on a webpage

Question

I need to code a bot that needs to do the following:

Go to a jsp page and search for something by:

• 1: writing something on a search box
• 2: clicking the search button(submit button)
• 3: clicking one of the the resulting buttons/links(same jsp page with different output)
• 4: get the entire html of the new page(same jsp page with different output)

The 4th one can be done with screen scraping and I do not think I need help with it. But I need some guidance to do the options from 1 to 3. Any links or just some keyword that will help me google to learn about it will be appreciated. I plan to do this with java.

Answer 1

All you need is HTMLUnit

This is an extract from its description

HtmlUnit is a "GUI-Less browser for Java programs". It models HTML documents and provides an API that allows you to invoke pages, fill out forms, click links, etc... just like you do in your "normal" browser.

P.S.: Had used it to build a web scraping project ;)

Answer 2

Maybe it's not what you want, but you can try selenium : http://seleniumhq.org/

It's a web application testing system.