dlopen doesn't respect `RTLD_LOCAL`?

Question

I have A.so, which links to a particular versioned libstdc++.so.6 at its own directory (via rpath set to $ORIGIN).

If I dlopen A.so alone, it works fine.

If I dlopen my system's libstdc++.so.6 (which is of different version) in RTLD_LOCAL mode, and then dlopen A.so

OSError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by A.so)

Why doesn't dlopen respect RTLD_LOCAL?

Answer 1

Why doesn't dlopen respect RTLD_LOCAL

RTLD_LOCAL doesn't mean what you think it means. From man dlopen:

RTLD_LOCAL
          This is the converse of RTLD_GLOBAL, and the default if
          neither flag is specified.  Symbols defined in this shared
          object are not made available to resolve references in
          subsequently loaded shared objects.

Note that this says nothing about which library the loader is loading.

The loader will never load more than one instance of a given SONAME (unless you use a different linker scope using dlmopen), so when you dlopen system libstdc++.so.6, that is the only libstdc++.so.6 you will ever get. When you later dlopen("A.so", ...), the runtime loader:

1. Looks for libraries that A.so depends on, discovers that libstdc++.so.6 is among them, and discovers that it already has libstdc++.so.6 loaded, so it doesn't search for, nor load another copy,
2. Looks for version symbols that A.so requires. It is at this point that you get the error, since A.so requires GLIBCXX_3.4.20 from libsstdc++.so.6, and since the already loaded system libstdc++.so.6 is older and doesn't provide GLIBCXX_3.4.20 version.
3. Since step #2 failed, your dlopen is rejected.

Note that you never get to resolving any symbols from libstdc++.so.6 (where RTLD_LOCAL would matter); you fail long before that.

Now, what you probably are trying to do is build A.so in such a way that it can be dynamically loaded into an arbitrary program, possibly one using an older version of libstdc++.so.6, without forcing the end-user to update system libstdc++.so.6. Unfortunately, this simply can not be done.