LWP refuses to connect via HTTPS

Question

I am running the following Perl snippet on Debian using Perl v5.14.2 and libwww-perl v6.04-1

my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new("GET", "https://google.com/");
my $rep = $ua->request($req);
print $rep->status_line;

This instantly returns "500 Can't connect to google.com:443". I have tried using LWP::Simple, Net::SSLeay, Crypt::SSLeay, etc. without any success.

Oddly enough, executing the same code on another Debian system running exactly the same Perl and LWP versions works.

So I thought, there is some error with the underlying system, but other applications - like cURL for any browser - are working fine.

Also, openssl s_client -connect google.com:443 returns Verify return code: 20 (unable to get local issuer certificate) on both systems.

Has anyone ever encountered this phenomenon and has a solution?

Answer 1

Instead of this:

$ua = LWP::UserAgent->new;

Try to use this:

$ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0 });

Answer 2

FYI for others battling LWP 500 erors:

Some LWP 500 errors are apparently caused by another type of SSL problem (which isn't solved with the verify_hostname setting). Instead, LWP may be communicating with the HTTPS server, but the response is not what LWP expects. In my case, the solution was to force SSLv3, by the following means:

my %ssl_options = (SSL_version => 'SSLv3');
$ua = LWP::UserAgent->new(ssl_opts => \%ssl_options), 

Also, for anyone trying to figure out what kind of 500 error you are getting, output this along with your error:

print $response->as_string;

For my problem (solved by setting SSLv3), my $response->as_string output included:

"SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert unexpected message LWP"

I'll also say that my code ran fine for years on an earlier version of Ubuntu. This problem only appeared once I upgraded Ubuntu. I've concluded that there are probably multiple ways that newer versions of LWP differ from older ones, so developers beware!

Good luck solving your LWP problem!