How do I use Perl's Google::Search to look for specific URLs?

Question

The Google::Search module, which is based on the AJAX Search API, doesn't seems to work very well, or it is just me?

For example, I use firefox to search google for: http://bloggingheads.tv/forum/member.php?u=12129

It brings results.

But when I use the module this way:

$google_search = Google::Search->Web ( q => "http://bloggingheads.tv/forum/member.php?u=12129" );
@result =  $google_search->all;

I get nothing in the array.

Any idea?

Seems like this API doesn't bring the same results like when searching manually, am I missing something?

Answer 1

I had a similar problem with cyrillic queries. Both Google::Search and REST::Google from CPAN didn't work for me - they were giving back fewer or no results compared to manual test.

Eventually I wrote a scraping module using WWW::Mechanize and HTML::TreeBuilder.

Here's a sample to get result stats:

my $tree = HTML::TreeBuilder->new_from_content($content);

if (my $div = $tree->look_down(_tag => 'div', id => 'resultStats')) {
    my $stats = $div->as_text();
}
else { warn "no stats" }