How can I get LWP to validate SSL server certificates?

Question

How can I get LWP to verify that the certificate of the server I'm connecting to is signed by a trusted authority and issued to the correct host? As far as I can tell, it doesn't even check that the certificate claims to be for the hostname I'm connecting to. That seems like a major security hole (especially with the recent DNS vulnerabilities).

Update: It turns out what I really wanted was HTTPS_CA_DIR, because I don't have a ca-bundle.crt. But HTTPS_CA_DIR=/usr/share/ca-certificates/ did the trick. I'm marking the answer as accepted anyway, because it was close enough.

Update 2: It turns out that HTTPS_CA_DIR and HTTPS_CA_FILE only apply if you're using Net::SSL as the underlying SSL library. But LWP also works with IO::Socket::SSL, which will ignore those environment variables and happily talk to any server, no matter what certificate it presents. Is there a more general solution?

Update 3: Unfortunately, the solution still isn't complete. Neither Net::SSL nor IO::Socket::SSL is checking the host name against the certificate. This means that someone can get a legitimate certificate for some domain, and then impersonate any other domain without LWP complaining.

Update 4: LWP 6.00 finally solves the problem. See my answer for details.

Answer 1

This long-standing security hole has finally been fixed in version 6.00 of libwww-perl. Starting with that version, by default LWP::UserAgent verifies that HTTPS servers present a valid certificate matching the expected hostname (unless $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} is set to a false value or, for backwards compatibility if that variable is not set at all, either $ENV{HTTPS_CA_FILE} or $ENV{HTTPS_CA_DIR} is set).

This can be controlled by the new ssl_opts option of LWP::UserAgent. See that link for details on how the Certificate Authority certificates are located. But be careful, the way LWP::UserAgent used to work, if you provide a ssl_opts hash to the constructor, then verify_hostname defaulted to 0 instead of 1. (This bug was fixed in LWP 6.03.) To be safe, always specify verify_hostname => 1 in your ssl_opts.

So use LWP::UserAgent 6; should be sufficient to have server certificates validated.

Answer 2

There are two means of doing this depending on which SSL module you have installed. The LWP docs recommend installing Crypt::SSLeay. If that's what you've done, setting the HTTPS_CA_FILE environment variable to point to your ca-bundle.crt should do the trick. (the Crypt::SSLeay docs mentions this but is a bit light on details). Also, depending on your setup, you may need to set the HTTPS_CA_DIR environment variable instead.

Example for Crypt::SSLeay:

  use LWP::Simple qw(get); $ENV{HTTPS_CA_FILE} = "/path/to/your/ca/file/ca-bundle"; $ENV{HTTPS_DEBUG} = 1;  print get("https://some-server-with-bad-certificate.com");  __END__ SSL_connect:before/connect initialization SSL_connect:SSLv2/v3 write client hello A SSL_connect:SSLv3 read server hello A SSL3 alert write:fatal:unknown CA SSL_connect:error in SSLv3 read server certificate B SSL_connect:error in SSLv3 read server certificate B SSL_connect:before/connect initialization SSL_connect:SSLv3 write client hello A SSL_connect:SSLv3 read server hello A SSL3 alert write:fatal:bad certificate SSL_connect:error in SSLv3 read server certificate B SSL_connect:before/connect initialization SSL_connect:SSLv2 write client hello A SSL_connect:error in SSLv2 read server hello B  

Note that get doesn't die, but it does return an undef.

Alternatively, you can use the IO::Socket::SSL module (also available from the CPAN). To make this verify the server certificate you need to modify the SSL context defaults:

  use IO::Socket::SSL qw(debug3); use Net::SSLeay; BEGIN {     IO::Socket::SSL::set_ctx_defaults(         verify_mode => Net::SSLeay->VERIFY_PEER(),         ca_file => "/path/to/ca-bundle.crt",       # ca_path => "/alternate/path/to/cert/authority/directory"     ); } use LWP::Simple qw(get);  warn get("https:://some-server-with-bad-certificate.com");  

This version also causes get() to return undef but prints a warning to STDERR when you execute it (as well as a bunch of debugging if you import the debug* symbols from IO::Socket::SSL):

  % perl ssl_test.pl DEBUG: .../IO/Socket/SSL.pm:1387: new ctx 139403496 DEBUG: .../IO/Socket/SSL.pm:269: socket not yet connected DEBUG: .../IO/Socket/SSL.pm:271: socket connected DEBUG: .../IO/Socket/SSL.pm:284: ssl handshake not started DEBUG: .../IO/Socket/SSL.pm:327: Net::SSLeay::connect -> -1 DEBUG: .../IO/Socket/SSL.pm:1135: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed  DEBUG: .../IO/Socket/SSL.pm:333: fatal SSL error: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed DEBUG: .../IO/Socket/SSL.pm:1422: free ctx 139403496 open=139403496 DEBUG: .../IO/Socket/SSL.pm:1425: OK free ctx 139403496 DEBUG: .../IO/Socket/SSL.pm:1135: IO::Socket::INET configuration failederror:00000000:lib(0):func(0):reason(0) 500 Can't connect to some-server-with-bad-certificate.com:443 (SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed)