Access website - WWW::Mechanize

Question

I try to use the code as below to get the website htm source and it works. However, I cannot get the result when I visit the website http://reserve.apple.com/WebObjects/ProductReservation.woa/wa/reserveProduct by using code as below. But, I can access this page by using browser properly. Would you give me some hints or tips to fix this problem? Thank you.

#!/usr/bin/perl

use strict;
use warnings;

# create a new browser
use WWW::Mechanize;
my $browser = WWW::Mechanize->new();

# tell it to get the main page

my $sURL = 'http://www.apple.com';

#my $sURL = 'http://reserve.apple.com/WebObjects/ProductReservation.woa/wa/reserveProduct';

$browser->get($sURL);

print $browser->content;

exit(0);

Answer 1

It's a strange behavior, but site at url you want to retrieve requires following headers to be defined: Accept, Accept-Encoding, Accept-Language, Accept-Charset, Cookie.

Otherwise server does not respond at all.

You can easy do this just inserting following code before your "get" request:

$browser->add_header(
    "Accept"          => "",
    "Accept-Encoding" => "",
    "Accept-Language" => "",
    "Accept-Charset"  => "",
    "Cookie"          => ""
);

Instead of empty fields you can insert some real values, but this works too.