How to solve segmentation fault problems happening in support libraries?

Question

I have a very odd problem going on. I can replicate the problem by the following small sample code:

#include <openssl/ssl.h>
#include <openssl/err.h>

#include <iostream>

void printSSLErrors()
{
    int l_err = ERR_get_error();
    while(l_err!=0)
    {
      std::cout << "SSL ERROR: " << ERR_error_string(l_err, NULL) << std::endl;
      l_err = ERR_get_error();
    }
}

int main(int argc, char* argv[]) {

    SSL_library_init();
    SSL_load_error_strings();

    // context
    SSL_CTX* mp_ctx;

    if(!(mp_ctx = SSL_CTX_new(SSLv23_server_method())))
    {
      printSSLErrors();
      return 0;
    }

    std::cout << "CTX created OK" << std::endl;

    // set certificate and private key
    if(SSL_CTX_use_certificate_file(mp_ctx, argv[1], SSL_FILETYPE_PEM)!=1)
    {
      printSSLErrors();
      return 0;
    }

    std::cout << "Certificate intialised OK" << std::endl;

    if(SSL_CTX_use_PrivateKey_file(mp_ctx, argv[2], SSL_FILETYPE_PEM)!=1)
    {
      printSSLErrors();
      return 0;
    }

    std::cout << "Key intialised OK" << std::endl;

    SSL_CTX_free(mp_ctx);
    ERR_free_strings();
}

This program works as expected when I compile it and link it using -lssl. The problem however is that the openssl routines are part of an application that also links in the mysqlclient libraries. I now recompile the above code with -lssl -lmysqlclient (note that I don't include or use anything from that library here). If I execute the program again I get a segmentation fault in the open ssl library. The most I can pull out of gdb is:

[Thread debugging using libthread_db enabled]
[New Thread -1208158528 (LWP 32359)]
CTX created OK
Certificate intialised OK

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1208158528 (LWP 32359)]
0x001b1213 in X509_get_pubkey () from /lib/libcrypto.so.4

(gdb) backtrace
#0  0x001b1213 in X509_get_pubkey () from /lib/libcrypto.so.4
#1  0x00de8a6c in SSL_rstate_string () from /lib/libssl.so.4
#2  0x086f2258 in ?? ()
#3  0xbffceb64 in ?? ()
#4  0x086f1130 in ?? ()
#5  0xbffceaa8 in ?? ()
#6  0x086f2258 in ?? ()
#7  0x086f0d90 in ?? ()
#8  0x00df4858 in ?? () from /lib/libssl.so.4
#9  0x086f2258 in ?? ()
#10 0x086f1130 in ?? ()
#11 0xbffceaa8 in ?? ()
#12 0x00de9d50 in SSL_CTX_use_PrivateKey_file () from /lib/libssl.so.4
Previous frame inner to this frame (corrupt stack?)
(gdb) frame 0
#0  0x001b1213 in X509_get_pubkey () from /lib/libcrypto.so.4

For some reason this only happens when I use mysqlclient v 15 and not with mysqlclient v 16. This is probably too obscure for anyone to solve, but some comments on how linking against a dynamic library that the code itself doesn't even use can cause these errors would be very helpful.

The system is:

RHEL ES4, gcc 3.4.6, openssl-0.9.7a, MySQL-5.11

Any thoughts?

Edit: Here is the output to possibly clarify things a little more:

[Lieuwe ~]$ c++ openssl_test.cpp -lssl -o ssltest 
[Lieuwe ~]$ ./ssltest /etc/httpd/conf/certs/test.crt /etc/httpd/conf/certs/test.key 
CTX created OK
Certificate intialised OK
Key intialised OK
[Lieuwe ~]$ c++ openssl_test.cpp -lmysqlclient -lssl -o ssltest 
[Lieuwe ~]$ ./ssltest /etc/httpd/conf/certs/test.crt /etc/httpd/conf/certs/test.key 
CTX created OK
Certificate intialised OK
Segmentation fault (core dumped)
[Lieuwe ~]$ 

Note that for this purpose I use the crt and key file that the apache server also uses (and work)

Edit 2: Here is the (relevant?) output of valgrind for the program

CTX created OK
--5429-- REDIR: 0x5F6C80 (memchr) redirected to 0x4006184 (memchr)
Certificate intialised OK
==5429== Invalid read of size 4
==5429==    at 0xCF4205: X509_get_pubkey (in /lib/libcrypto.so.0.9.7a)
==5429==    by 0xDE8A6B: (within /lib/libssl.so.0.9.7a)
==5429==    by 0xDE9D4F: SSL_CTX_use_PrivateKey_file (in /lib/libssl.so.0.9.7a)
==5429==    by 0x8048C77: main (in /home/liwu/ssltest)
==5429==  Address 0x4219940 is 0 bytes inside a block of size 84 free'd
==5429==    at 0x4004EFA: free (vg_replace_malloc.c:235)
==5429==    by 0xC7FD00: CRYPTO_free (in /lib/libcrypto.so.0.9.7a)
==5429==    by 0xCE53A7: (within /lib/libcrypto.so.0.9.7a)
==5429==    by 0xCE5562: ASN1_item_free (in /lib/libcrypto.so.0.9.7a)
==5429==    by 0xCE0560: X509_free (in /lib/libcrypto.so.0.9.7a)
==5429==    by 0xDE979E: SSL_CTX_use_certificate_file (in /lib/libssl.so.0.9.7a)
==5429==    by 0x8048C23: main (in /home/liwu/ssltest)
==5429== 
==5429== Invalid read of size 4
==5429==    at 0xCD4A5F: EVP_PKEY_copy_parameters (in /lib/libcrypto.so.0.9.7a)
==5429==    by 0xDE8A7C: (within /lib/libssl.so.0.9.7a)
==5429==    by 0xDE9D4F: SSL_CTX_use_PrivateKey_file (in /lib/libssl.so.0.9.7a)
==5429==    by 0x8048C77: main (in /home/liwu/ssltest)
==5429==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==5429== 
==5429== Process terminating with default action of signal 11 (SIGSEGV)
==5429==  Access not within mapped region at address 0x0
==5429==    at 0xCD4A5F: EVP_PKEY_copy_parameters (in /lib/libcrypto.so.0.9.7a)
==5429==    by 0xDE8A7C: (within /lib/libssl.so.0.9.7a)
==5429==    by 0xDE9D4F: SSL_CTX_use_PrivateKey_file (in /lib/libssl.so.0.9.7a)
==5429==    by 0x8048C77: main (in /home/liwu/ssltest)
==5429== 

Answer 1

I would suggest running your program under Valgrind. Valgrind is intended to provide help with exactly this kind of problem and it is generally much easier to use than a debugger.

If I were to hazard a guess, I would first suspect a memory error in your application (or, less likely, in one of the shared libraries) that is sensitive to the memory layout of the resulting executable. Adding a new shared library or, say, enabling debugging options could very well make the problem appear or disappear for no apparent reason.