I am building a shared library on Linux, which serves as a "plugin" to some software (to be specific, it extends Mathematica).
I find that if I build on Ubuntu 16.04, the resulting library does not work on RHEL 7.6. However, if I build on RHEL 7.6, the library works both on RHEL and Ubuntu.
By "does not work", I mean that Mathematica refuses to load it, but it only gives a generic and unuseful "failed to load" error message.
I have eliminated a number of factors that could break compatibility, and I cannot find any more. This question is about what else might affect compatibility than what I list below.
The library is written in a mix of C and C++, but it exports a C interface. It is built with -static-libstdc++
and -static-libgcc
. If I use ldd
on the .so
file, the only dependencies it lists are:
linux-vdso.so.1 => (0x00007ffc757b9000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa286e62000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa287854000)
One potential source of incompatibility is the glibc
version. I looked at the symbols in the library using nm -gC
, and the highest GLIBC version reference I see when I build on Ubuntu is 2.14. RHEL 7.6 has glibc 2.17, i.e. newer than 2.14. Thus I do not believe that the incompatibility is due to glibc.
What else is there that can cause a shared object compiled on Ubuntu 16.04 not to load on RHEL 7.6?
Update: I managed to coax Mathematica to give a more descriptive error (it was a not very well documented feature), so I have a concrete error message. The same could also be seen with @Ctx's suggestion to set LD_DEBUG=all
.
The error is:
IGraphM.so: undefined symbol: _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE
(IGraphM.so
is my library.)
This function would seem to be part of libstdc++ unless I am mistaken. Why does this error occur if I specified -static-libstdc++
and verified that ldd
does not list libstdc++?
Update 2:
Per the advice by SergeyA and this QA, I compiled after defining _GLIBCXX_USE_CXX11_ABI=0
. This does fix the incompatibility.
But I still do not understand why. The error message complains about a missing symbol. Where is this symbol normally loaded from? I was under the impression that if I use -static-libstdc++
, then it should be contained within my library. This seems to be wrong.
While I seem to have a practical solution for the incompatibility for this specific case, I would appreciate some explanations so in the future I can solve similar problems on my own.
However, as a rule, you'll find that Linux distributions are not binary compatible. Whether you're writing your code or purchasing code from a vendor, binary compatibility is something you need to look out for given the complex field of Linux distributions.
Shared libraries are normally split into several binary packages. The SONAME symlink is installed by the runtime shared library package, and the bare . so symlink is installed in the development package since it's only used when linking binaries or shared libraries.
Shared libraries are the most common way to manage dependencies on Linux systems. These shared resources are loaded into memory before the application starts, and when several processes require the same library, it will be loaded only once on the system. This feature saves on memory usage by the application.
Binary-code compatibility (binary compatible or object-code-compatible) is a property of a computer system, meaning that it can run the same executable code, typically machine code for a general-purpose computer CPU, that another computer system can run.
I can't explain why your .so library doesn't link all the used symbols statically (and instead leaves them as undefined), but I can offer practical suggestion on how to fix the issue at hand.
You can stop linking libstdc++
statically into your plugin, and instead use the one available to host system. This doesn't work for you because of ABI incompatibility between build and target platforms. You can downgrade an ABI in use for your plugin by specifying macro _GLIBCXX_USE_CXX11_ABI=0
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With