Using C++11 multithreading in shared library loaded by program without thread support

Question

I am currently trying to use C++11 multithreading in a shared library that is loaded into the main program (written in C) on Linux. This is part of a big simulation program and I cannot change anything about the library loading or change the main program in general.

The main program is compiled with gcc 4.1.2 and I don't have the sources for it (I cannot recompile it with gcc 4.8.2).

The shared library is compiled with gcc 4.8.2 in order to use C++11 multithreading. I am passing the compiler commands

-pthread -lpthread -std=c++11

as explained in What is the correct link options to use std::thread in GCC under linux?

Compiling a standalone test program with this configuration ("-pthread -std=c++11" and gcc 4.8) works correctly on my system. But when I start the program loading the shared library I get an exception:

Caught std::exception!
Exception Message: Enable multithreading to use std::thread: Operation not permitted

Terminating...

Using the -pthread and -lpthread (Edit: and also only -pthread without -lpthread) compile parameter doesn't work. The compiler arguments are (I am using the cook build system):

-pthread -std=c++11 -fmessage-length=0 -fPIC -Wchar-subscripts ...(lots of -W* here)
... -Wunused-variable -m64 -D__64BIT__ -pthread -lpthread

and the linker arguments (duplicate parameters due to the build system):

-pthread -lpthread -std=c++11 -pthread -lpthread -std=c++11 -shared -fPIC -Wl,-Bsymbolic -Wl,--allow-shlib-undefined -pthread -lpthread

calling ldd on my library gives the following output

$ ldd calc3/build/amd64_linux26_RH5/library.so
    linux-vdso.so.1 =>  (0x00007fff4d1fd000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ae6ec124000)
    libstdc++.so.6 => /afs/bb/data/d6833/util/gcc_482/lib64/libstdc++.so.6 (0x00002ae6ec340000)
    libm.so.6 => /lib64/libm.so.6 (0x00002ae6ec655000)
    libgcc_s.so.1 => /afs/bb/data/d6833/util/gcc_482/lib64/libgcc_s.so.1 (0x00002ae6ec8d8000)
    libc.so.6 => /lib64/libc.so.6 (0x00002ae6ecaef000)
    /lib64/ld-linux-x86-64.so.2 (0x00000032cb400000)

and on the main program

$ ldd .../bin-64/main_program
    linux-vdso.so.1 =>  (0x00007fff64595000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00000032cc000000)
    libz.so.1 => /usr/lib64/libz.so.1 (0x00000032cc800000)
    libc.so.6 => /lib64/libc.so.6 (0x00000032cb800000)
    /lib64/ld-linux-x86-64.so.2 (0x00000032cb400000)

The pthread library is linked to my shared library but not to the main program. This answer states that you have to link pthreads to the main program but the 2nd comment on this answer (by @R..) says it isn't necessary (which sounds logically).

Unfortunately I don't know anything about the loading mechanics of the whole system, except that my library is using another C++ library as API.

Note that other C++11 features do work (and libstdc++.so is in the dependencies of my library) but C++11 multithreading is not (although libpthread.so is also in the dependencies of my library).

Using a threading class from a library contained in the program itself is working (and this thread class seems to use pthreads too).

I have also tried to use -fabi-version=0 or -fabi-version=2 because the main program is compiled with gcc 4.1.2 with my library but it didn't change anything.

Is there anything I have overlooked or a compiler option I can use to make it work? Or does it seem to be a problem of my program environment? Any ideas are welcome.

Edit:

I tried using -Wl,-no-as-needed (as suggested in the comments) but it unfortunately didn't change anything.

Using clang 3.5 instead of gcc 4.8 also didn't work.

Creating a small test application which loads a shared library (like in the answer below by @chill) works (even without the compiler flag) as long as I use gcc 4.8 or clang 3.5 for both main application and shared library. When using gcc 4.1 for the main program however the main program even fails to load the library (which is working in my 'real' application). I think there might be a problem with the differen ABIs of the compilers.

Using pthreads directly from pthread.h seems to work (although the program currently terminates on pthread_join without error message, but I'm still testing there...)

Edit 2:

Running the 'test program' with LD_LIBRARY_PATH=$(pwd):$LD_LIBRARY_PATH (because the gcc 4.8 library paths need to be in there too, thanks @MvG) did run the program but crashed again with the Enable multithreading to use std::thread: Operation not permitted exception.

I checked all other libraries that are loaded (found them with strace ./main_program 2>&1 | grep '^open(".*\.so"' [see here]) and checked them all with ldd. They all depend on the same libraries (with the same paths). ldd outputs (on all of them):

linux-vdso.so.1 =>  (0x00007fff4d3fd000)
libstdc++.so.6 => /afs/bb/data/d6833/util/gcc_482/lib64/libstdc++.so.6 (0x00002ade28774000)
libm.so.6 => /lib64/libm.so.6 (0x00002ade28ab0000)
libgcc_s.so.1 => /afs/bb/data/d6833/util/gcc_482/lib64/libgcc_s.so.1 (0x00002ade28d33000)
libc.so.6 => /lib64/libc.so.6 (0x00002ade28f49000)
/lib64/ld-linux-x86-64.so.2 (0x00000032ea200000)

(that they all don't depend on libpthread.so.0 except my library and one other (but it's the same /lib64/libpthread.so.0))

Some libraries do have more dependencies (which don't seem thread-related) but there don't seem to be any 'conflicting' dependencies (there are no dependencies to different versions/paths of the same library in any of these libraries).

Answer 1

In thread.cc you can read that this exception is generated if __gthread_active_p returns false. That call simply checks whether or not a given symbol is available. The symbol in question is a weak symbol: it does not have to be present, but its presence is checked to decide whether or not threading is supported.

But what does presence of a symbol mean? In this case it means that the symbol is in the list of symbol tables which the library in question (libgcc_s.so.1 in my case) searches for symbol definitions. That includes symbols exported by the application itself, but also symbols exported by all the libraries loaded before it. However, it does not include libraries loaded afterwards. Unfortunately, if libgcc is loaded before libpthread, then the symbol is not available on its search domain. So it reports threading as unsupported. I guess you have some other C++ module loaded before the multi-threaded one, so you encounter this problem.

One solution which works in my reproducing example is setting LD_PRELOAD=/lib64/libpthread.so.0 in the environment used to call the binary. That loads libpthread up front, so its symbols are available to satisfy the weak symbol linkage. This won't work for setuid/setgid binaries, and might be considered an ugly hack in other cases as well, so I'm interested in cleaner solutions. Nevertheless, this gets the job done most of the time.