Digital Signing using certificate and key from USB token

Question

I want to sign a file using the user's key and certificate from a USB token (dongle).

I have been searching on this for sometime, on stackoverflow and other sites, but didn't get anything useful apart from some good capabilities in .NET framework (which I am not using).

It seems that since the key is not exposed, the encryption is done by the hardware itself. Does that mean that each hardware manufacturer provides his own APIs and that there is no generic method to address this problem?

Also, I read that once the token is plugged into the computer, its certificate is loaded into the system store. Is it possible to use the certificate from the store? How can such a certificate be identified and accessed among others in the store? and what about the private key?

I have used OpenSSL for digital signing when the certificate can be extracted from a .p12 or .pfx file.

Correct me if I am wrong somewhere, I am new to this topic.

Answer 1

I don't know that I'd characterize any aspect of OpenSSL engines as "fairly simple". The command-line version is chaotic, and I could only figure out what the command line was doing after doing it in code myself. The order of operations and lifetime are not called out very well (and I still don't fully know what those are, but I have my system running and no longer leaking memory, so yay.)

I've put up functioning versions on github: https://github.com/tkil/openssl-pkcs11-samples

Here is a guided tour through the relevant portions of tok-sign.c:

First, some helpers:

#define FAIL( msg, dest )                      \
    do {                                       \
        fprintf( stderr, "error: " msg "\n" ); \
        goto dest;                             \
    } while ( 0 )

/* mandatory is "not optional"... */
const int CMD_MANDATORY = 0;

Probably the oddest thing about the dynamic engine is that it's really a meta-engine: you feed it various parameters, and when you feed it LOAD, it loads the dynamic library and makes a new engine available. It's straightforward in code, once you know the correct order of operations. Here, we get access to the dynamic engine, configure it, and then ask it to bring in the pkcs11 engine:

ENGINE_load_dynamic();
ENGINE * dyn = ENGINE_by_id( "dynamic" );
if ( ! dyn )
    FAIL( "retrieving 'dynamic' engine", free_out_sig_file );

// this is the bridge between OpenSSL and any generic PCKS11 provider:
char * engine_pkcs11_so = "/opt/crypto/lib/engines/engine_pkcs11.so";

if ( 1 != ENGINE_ctrl_cmd_string( dyn, "SO_PATH", engine_pkcs11_so, CMD_MANDATORY ) )
    FAIL( "dyn: setting so_path <= 'engine_pkcs11.so'", free_dyn );

if ( 1 != ENGINE_ctrl_cmd_string( dyn, "ID", "pkcs11", CMD_MANDATORY ) )
    FAIL( "dyn: setting id <= 'pkcs11'", free_dyn );

if ( 1 != ENGINE_ctrl_cmd( dyn, "LIST_ADD", 1, NULL, NULL, CMD_MANDATORY ) )
    FAIL( "dyn: setting list_add <= 1", free_dyn );

if ( 1 != ENGINE_ctrl_cmd( dyn, "LOAD", 1, NULL, NULL, CMD_MANDATORY ) )
    FAIL( "dyn: setting load <= 1", free_dyn );

At this point, if all those calls succeeded, your OpenSSL instance now has access to a new engine called "pkcs11". Now we need to get access to that new engine, configure it correctly, and let it initialize itself:

ENGINE * pkcs11 = ENGINE_by_id( "pkcs11" );
if ( ! pkcs11 )
    FAIL( "pkcs11: unable to get engine", free_dyn );

// this is the actual pkcs11 provider we're using.  in this case, it's
// from the OpenSC package.
char * opensc_pkcs11_so = "/opt/crypto/lib/opensc-pkcs11.so";

if ( 1 != ENGINE_ctrl_cmd_string( pkcs11, "MODULE_PATH", opensc_pkcs11_so, CMD_MANDATORY ) )
    FAIL( "setting module_path <= 'opensc-pkcs11.so'", free_pkcs11 );

if ( 1 != ENGINE_ctrl_cmd_string( pkcs11, "PIN", key_pin, CMD_MANDATORY ) )
    FAIL( "setting pin", free_pkcs11 );

if ( 1 != ENGINE_init( pkcs11 ) )
    FAIL( "pkcs11: unable to initialize engine", free_pkcs11 );

Now that we have access to the token, we can get the private key for use in OpenSSL operations:

EVP_PKEY * key = ENGINE_load_private_key( pkcs11, key_id, NULL, NULL );
if ( ! key )
    FAIL( "reading private key", free_pkcs11 );

However, I couldn't figure out how to extract the corresponding certificate through the ENGINE interface, so I went directly to LibP11:

PKCS11_CTX * p11_ctx = PKCS11_CTX_new();
if ( ! p11_ctx )
    FAIL( "opening pkcs11 context", free_key );

if ( 0 != PKCS11_CTX_load( p11_ctx, opensc_pkcs11_so ) )
    FAIL( "unable to load module", free_p11_ctx );

PKCS11_SLOT * p11_slots;
unsigned int num_p11_slots;
if ( 0 != PKCS11_enumerate_slots( p11_ctx, &p11_slots, &num_p11_slots ) )
    FAIL( "enumerating slots", free_p11_ctx_module );

PKCS11_SLOT * p11_used_slot =
  PKCS11_find_token( p11_ctx, p11_slots, num_p11_slots );
if ( ! p11_used_slot )
    FAIL( "finding token", free_p11_slots );

PKCS11_CERT * p11_certs;
unsigned int num_p11_certs;
if ( 0 != PKCS11_enumerate_certs( p11_used_slot->token, &p11_certs, &num_p11_certs ) )
    FAIL( "enumerating certs", free_p11_slots );

Finally, we start collecting data for the CMS_Sign request. We look at all the certificates on the token, pick out the one corresponding to the private key, then store the rest in an OpenSSL STACK_OF(X509):

STACK_OF(X509) * extra_certs = sk_X509_new_null();
if ( ! extra_certs )
    FAIL( "allocating extra certs", free_p11_slots );

X509 * key_cert = NULL;
for ( unsigned int i = 0; i < num_p11_certs; ++i )
{
    PKCS11_CERT * p11_cert = p11_certs + i;

    if ( ! p11_cert->label )
        continue;

    // fprintf( stderr, "p11: got cert label='%s', x509=%p\n",
    //         p11_cert->label, p11_cert->x509 );

    if ( ! p11_cert->x509 )
    {
        fprintf( stderr, "p11: ... no x509, ignoring\n" );
        continue;
    }

    const char * label = p11_cert->label;
    const unsigned int label_len = strlen( label );

    if ( strcmp( label, key_label ) == 0 )
    {
        // fprintf( stderr, "p11: ... saving as signing cert\n" );
        key_cert = p11_cert->x509;
    }
    else if ( strncmp( label, "encrypt", 7 ) == 0 &&
              label_len == 8 &&
              '0' <= label[7] && label[7] <= '3' )
    {
        // fprintf( stderr, "p11: ... ignoring as encrypting cert\n" );
    }
    else
    {
        // fprintf( stderr, "p11: ... saving as extra cert\n" );
        if ( ! sk_X509_push( extra_certs, p11_cert->x509 ) )
            FAIL( "pushing extra cert", free_extra_certs );
    }
}

if ( ! key_cert )
    FAIL( "finding signing cert", free_extra_certs );

Finally, we can sign the data, then output the signature in a DER-formatted file:

CMS_ContentInfo * ci = CMS_sign( key_cert, key, extra_certs, in_data_file,
                                 CMS_DETACHED | CMS_BINARY );
if ( ! ci )
    FAIL( "could not create signing structure", free_extra_certs );

if ( 1 != i2d_CMS_bio( out_sig_file, ci ) )
       FAIL( "could not write signature in DER", free_ci );

After that, it's just cleanup:

free_ci:
    CMS_ContentInfo_free( ci );

free_extra_certs:
    /* these certs are actually "owned" by the libp11 code, and are
     * presumably freed with the slot or context. */
    sk_X509_free( extra_certs );

free_p11_slots:
    PKCS11_release_all_slots( p11_ctx, p11_slots, num_p11_slots );

free_p11_ctx_module:
    PKCS11_CTX_unload( p11_ctx );

free_p11_ctx:
    PKCS11_CTX_free( p11_ctx );

free_key:
    EVP_PKEY_free( key );

free_pkcs11:
    ENGINE_free( pkcs11 );

free_dyn:
    ENGINE_free( dyn );

Answer 2

You can do this by using the engines of OpenSSL - and providing it with a PKCS#11 engine.

This can be done from the command line (the -engine flag) or by setting the engine. apps.c in /apps/ of the openssl distribution has good examples.

A typical invocation form the command line looks like

usr/bin/openssl << EOM
engine dynamic -pre SO_PATH:/Library/OpenSC/lib/engines/engine_pkcs11.so  -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre MODULE_PATH:opensc-pkcs11.so
req -engine pkcs11 -batch -subj "/CN=moi" -new -key slot_$SLOT-id_$KID -keyform engine -x509 -out cert.pem -text
EOM

which creates&signs a request on the device.

To sign something:

${OPENSSL} << EOM || exit 1
engine -vvvv dynamic \
    -pre SO_PATH:/Library/OpenSC/lib/engines/engine_pkcs11.so  \
    -pre ID:pkcs11 \
    -pre LIST_ADD:1 \
    -pre LOAD \
    -pre MODULE_PATH:$PKCS \
    -pre PIN:$PIN  \
    -pre VERBOSE 

x509 -engine pkcs11 -req -in req.csr -out signed.pem \
    -CAfile $CA \
    -keyform engine -key $SLOT:$CAKID \
    -cert $CAKID.pem

do note though that for the latter an unpatched openssl does not allow for referencing of the cert on the card (only the key). So one needs to extract this first and store as a file.

and to use a key on the card to connect with client auth to a server:

${OPENSSL} << EOM || exit 1
engine -vvvv dynamic \
    -pre SO_PATH:/Library/OpenSC/lib/engines/engine_pkcs11.so  \
    -pre ID:pkcs11 \
    -pre LIST_ADD:1 \
    -pre LOAD \
    -pre MODULE_PATH:$PKCS \
    -pre PIN:$PIN  \
    -pre VERBOSE 

s_client -engine pkcs11 -connect localhost:1443 \
    -CAfile $CA \
    -keyform engine -key $SLOT:$KID \
    -cert $KID.pem

EOM

this is fairly simple to translate to native code - just search for engine in the apps.c in openssl apps.c file - to see how this is done. In most cases it is some

pkey = ENGINE_load_private_key(e, file,

as opposed to the old

BIO_read_filename(key,file)
pkey=d2i_PrivateKey_bio(key, NULL);