Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I handle Javascript in a Perl web crawler?

I would like to crawl a website, the problem is, that its full of JavaScript things, such as buttons and such that when they are pressed, they do not change the URL, but the data on the page is changed.

Usually I use LWP / Mechanize etc to crawl sites, but neither support JavaScript. any idea?

like image 952
snoofkin Avatar asked Sep 22 '10 12:09

snoofkin


1 Answers

Another option might be Selenium with WWW::Selenium module

like image 170
erickb Avatar answered Sep 30 '22 17:09

erickb