How can I get started with web page scraping using Perl?

Question

I am interested in learning Perl. I am using Learning Perl books and cpan's web-sites for reference.

I am looking forward to do some web/text scraping application using Perl to apply whatever I have learnt.

Please suggest me some good options to begin with.

(this is not a homework. want to do something in Perl that would help me exploit basic Perl features)

Answer 1

If the web pages you want to scrape require JavaScript to function properly, you are going to need more than what WWW::Mechanize can provide you. You might even have to resort to controlling a specific browser via Perl (e.g. using Win32::IE::Mechanize or WWW::Mechanize::Firefox).

I haven't tried it, but there is also WWW::Scripter with the WWW::Scripter::Plugin::JavaScript plugin.