C++ force unloading shared library

Question

I'm trying to create an application which reloads a shared library multiple times. But at some point in time, dlmopen fails with error

/usr/lib/libc.so.6: cannot allocate memory in static TLS block

Here is the minimal code reproducing this issue:

#include <dlfcn.h>
#include <cstdio>
#include <vector>

int main() {
  for (int i = 0; i < 100; ++i) {
    void *lib_so = dlmopen(LM_ID_NEWLM, "lib.so", RTLD_LAZY | RTLD_LOCAL);
    if (lib_so == NULL) {
      printf("Iteration %i loading failed: %s\n", i, dlerror());
      return 1;
    }
    dlclose(lib_so);
  }

  return 0;
}

And empty lib.cpp, compiled with

g++ -rdynamic -ldl -Wl,-R . -o test main.cpp
g++ -fPIC -shared lib.cpp -o lib.so

Update

It seems that it crashes even with one thread. The question is: how can I force a library unload or a destruction of unused namespaces created with LM_ID_NEWLM?

Answer 1

There is a built-in limit to the number of link map namespaces available to a process. This is rather poorly documented in the comment:

The glibc implementation supports a maximum of 16 namespaces

in the man page.

Once you create a link map namespace, there is no support for 'erasing' it via any APIs. This is just the way it's designed, and there's no real way to get around that without editing the glibc source and adding some hooks.

Using namespaces for reloading of a library is not actually reloading the library - you're simply loading a new copy of the library. This is one of the use cases of the namespaces - if you tried to dlopen the same library multiple times, you would get the same handle to the same library; however if you load the second instance in a different namespace, you won't get the same handle. If you want to accomplish reloading, you need to unload the library using dlclose, which will unload the library once the last remaining reference to the library has been released.

If you want to attempt to 'force unload' a library, then you could try issuing multiple dlclose calls until it unloads; however if you don't know what the library has done (e.g. spawned threads) there may be no way of preventing a crash in that case.