How to accept self-signed certificates with LWP::UserAgent

Question

I am attempting to set up a node.js server that uses HTTPS. I will then write a scripts in Perl to make a HTTPS request to the server and measure latency of the round trip.

Here is my node.js:

var express = require('express');
var https = require('https');
var fs = require('fs');

var key = fs.readFileSync('encrypt/rootCA.key');
var cert = fs.readFileSync('encrypt/rootCA.pem');

// This line is from the Node.js HTTPS documentation.
var options = {
  key: key,
  cert: cert
};

https.createServer(options, function (req, res) {
    res.writeHead(200);
    res.end("hello world - https\n");
}).listen(8088);

Key/cert generation was done as follows:

openssl genrsa -out rootCA.key 2048
openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem

This is my Perl script:

#!/usr/bin/perl
use LWP::UserAgent;


my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new(GET => 'https://127.0.0.1:8080');
my $res = $ua->request($req);

if ($res->is_success) {
  print $res->as_string;
} else {
  print "Failed: ", $res->status_line, "\n";
}

Returning the error:

Failed: 500 Can't verify SSL peers without knowing which Certificate Authorities to trust

The node.js documentation describes how to set up an HTTPS server but it is vague about generating primary cert and intermediate cert.

https://medium.com/netscape/everything-about-creating-an-https-server-using-node-js-2fc5c48a8d4e

Answer 1

The typical answer to such question is to disable certificate validation at all. But, this is totally insecure and essentially disables most of the protection offered by HTTPS. If validation is disabled completely a man in the middle attacker just can use an arbitrary certificate to intercept the connection and sniff and modify the data. Thus, don't do it.

The correct way to deal with such certificates is to add these certificates as trusted. This can be done with the SSL_ca_file argument:

my $ua = LWP::UserAgent->new;
$ua->ssl_opts(SSL_ca_file => 'rootCA.pem');
$ua->get('https://127.0.0.1:8080');

By explicitly trusting the self-signed server certificate as CA it will no longer throw "certificate verify failed".

But, unless your server certificate is actually issued to "127.0.0.1" you will now get "hostname verification failed" since the subject of the certificate does not match the domain of the URL. This can be fixed by setting the expected hostname:

my $ua = LWP::UserAgent->new;
$ua->ssl_opts(
    SSL_ca_file => 'rootCA.pem',
    SSL_verifycn_name => 'www.example.com',
);
$ua->get('https://127.0.0.1:8080');

Note that SSL_ca_file needs the self-signed certificate to have the CA flag set to true, i.e. that the certificate is a CA certificate which can be used to issue other certificates. If this is not the case with your certificate or if you just want to accept a specific certificate no matter if it is expired, revoked, does not match the hostname etc you can do the validation by using the fingerprint of the certificate.

my $ua = LWP::UserAgent->new;
$ua->ssl_opts(SSL_fingerprint => 'sha1$9AA5CFED857445259D90FE1B56B9F003C0187BFF')
$ua->get('https://127.0.0.1:8080');

The fingerprint here is the same you get with openssl x509 -noout -in rootCA.pem -fingerprint -sha1, only the algorithm added in front (sha1$...) and the colons removed.

Answer 2

To make LWP::UserAgent ignore server certificate use the following configuration:

my $ua = LWP::UserAgent->new;
$ua->ssl_opts(
    SSL_verify_mode => IO::Socket::SSL::SSL_VERIFY_NONE, 
    verify_hostname => 0
);