openssl debug information when using the library

Question

Is there a way to receive the same debug output when using library APIs as when using "s_client -debug"? (I am trying to debug handshake problems and it will not be practical to install the command-line tool on every system)

Answer 1

The -debug option provides debug output on various events which are hard coded into s_client. I think probably what you are most interested in is the bit that dumps the data read and written from the socket at each stage.

To do that you can use the following:

BIO_set_callback(sbio, callback);
BIO_set_callback_arg(sbio, (char *)outbio);

Where sbio is your socket BIO and outbio is a BIO for where you want the debug output to go. "callback" is a callback that actually dumps the output. The one that s_client uses looks like this:

long bio_dump_callback(BIO *bio, int cmd, const char *argp,
                       int argi, long argl, long ret)
{
    BIO *out;

    out = (BIO *)BIO_get_callback_arg(bio);
    if (out == NULL)
        return (ret);

    if (cmd == (BIO_CB_READ | BIO_CB_RETURN)) {
        BIO_printf(out, "read from %p [%p] (%lu bytes => %ld (0x%lX))\n",
                   (void *)bio, (void *)argp, (unsigned long)argi, ret, ret);
        BIO_dump(out, argp, (int)ret);
        return (ret);
    } else if (cmd == (BIO_CB_WRITE | BIO_CB_RETURN)) {
        BIO_printf(out, "write to %p [%p] (%lu bytes => %ld (0x%lX))\n",
                   (void *)bio, (void *)argp, (unsigned long)argi, ret, ret);
        BIO_dump(out, argp, (int)ret);
    }
    return (ret);
}

Another alternative is to use the SSL_trace() capability which gives you more human readable handshake information. To use that you must compile OpenSSL with the enable-ssl-trace option. Then you call:

SSL_set_msg_callback(con, SSL_trace);
SSL_set_msg_callback_arg(con, outbio);

Where con is the SSL connection, and outbio is where you want the debug output to go. SSL_trace is a built-in callback that can be used without further work.