SSL_CTX_use_PrivateKey_file() failed

Question

I'm writing a client application on Windows that establishes an SSL connection to a server, and the server requests client certificate for authentication. The server provides me a .pfx file, then I use openssl command line tool to get the certificate and the private key like this:

openssl pkcs12 -in filename.pfx -clcerts -nokeys -out cert.pem
openssl pkcs12 -in filename.pfx -nocerts -out key.pem

after that, I try to load the certificate and the private key with functions from openssl as below, but SSL_CTX_use_PrivateKey_file() always failed, the error message is "error:0906D06C:PEM routines:PEM_read_bio:no start line", I can't figure it out why, can anyone give me some enlightenment? Here is the code.

#include "openssl/ssl.h"
#include "openssl/err.h"
#include <stdio.h>
#include <string>

int InitClientCtx()
{
    OpenSSL_add_ssl_algorithms();

    SSL_CTX* m_pClientCtx;
    m_pClientCtx = SSL_CTX_new(SSLv23_method());

    if(!m_pClientCtx)
    {
        return -1;
    }

    ::SSL_CTX_set_options(m_pClientCtx, SSL_OP_ALL);  //for well-known bugs

    int nRet = 0;

    std::string sCertFilePath = "C:\\cert.pem";

    nRet = SSL_CTX_use_certificate_chain_file(m_pClientCtx, sCertFilePath.c_str());

    std::string sKeyPassWord = "123456";

    SSL_CTX_set_default_passwd_cb_userdata(m_pClientCtx, (void*)(sKeyPassWord.c_str()));

    std::string sKeyFilePath = "C:\\key.pem";

    // this method returned 0, which means it failed.
    nRet = SSL_CTX_use_PrivateKey_file(m_pClientCtx, sKeyFilePath.c_str(), SSL_FILETYPE_PEM);

    SSL_load_error_strings();
    unsigned long n = ERR_get_error();
    char buf[1024];
    printf("%s\n", ERR_error_string(n, buf));

    nRet = SSL_CTX_check_private_key(m_pClientCtx);
    if (nRet <= 0)
    {
        return -1;
    }

    /*std::string sCACertFilePath;

    nRet = SSL_CTX_load_verify_locations(m_pClientCtx, sCACertFilePath.c_str(), NULL);*/

    return 0;
}

int main()
{
    InitClientCtx();
    return 0;
};

Answer 1

In my case the error was because the PEM file did not contain both a key and a certificate.

Make sure your file contains both sections:

-----BEGIN PRIVATE KEY----- jhajk838383jks.....
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE----- yoe55wjcxnshre.....
-----END CERTIFICATE KEY-----

I catenated .key and .crt files I already had in my Apache configuration to make a .pem file.

The 'no start line' error is certainly misleading as you can have a perfectly good "BEGIN" line in your PEM file and still get the error.

Answer 2

The error error:0906D06C:PEM routines:PEM_read_bio:no start line is because in both the cert.pem as well as key.pem, don't start off with -----BEGIN CERTIFICATE----- and -----BEGIN ENCRYPTED PRIVATE KEY-----.

If you open up your cert.pem and key.pem in a text editor and yank off whatever is there before the BEGIN markers, you should be good.

When you create a certificate and a key pair using Certificate Signing Request, you won't get this additional information.

Answer 3

I've solved this problem myself. I generated the key.pem using OpenSSL for Windows, when the CMD prompts me to type in the pass phrase, I just typed a Enter since I needn't a pass phrase, but the key.pem was invalid(neither BEGIN nor END markers). When I generate the private key in Linux, the terminal prompts I must type a pass phrase and I do. Then I remove the key pass phrase using this command:

openssl rsa -in key.pem -out newkey.pem

After that, I open the key.pem in a text editor, it starts off with -----BEGIN RSA PRIVATE KEY----- and end up with -----END RSA PRIVATE KEY-----. And SSL_CTX_use_PrivateKey_file() just works fine!