Cannot load any more object with static TLS

Question

I have an application that use dlopen() to load additional modules. The application and modules are built on Ubuntu 12.04 x86_64 using gcc 4.6 but for i386 arch. The binaries are then copied to another machine with exactly same OS and work fine.

However if they are copied to Ubuntu 12.04 i386 then some (but not all) modules fail to load with the following message:

dlopen: cannot load any more object with static TLS

I would suspect that this is caused by the usage of __thread variables. However such variables are not used in the loaded modules - only in the loader module itself.

Can someone provide any additional info, what can be the reason?

I am reducing number of __thread variables and optimizing them (with -ftls-model etc), I'm just curious why it doesn't work on almost same system.

Answer 1

I would suspect that this is caused by the usage of __thread variables.

Correct.

However such variables are not used in the loaded modules - only in the loader module itself.

Incorrect. You may not be using __thread yourself, but some library you statically link in into your module is using them. You can confirm this with:

readelf -l /path/to/foo.so | grep TLS

what can be the reason?

The module is using -ftls-model=initial-exec, but should be using -ftls-model=global-dynamic. This most often happens when (some of) the code linked into foo.so is built without -fPIC.

Linking non-fPIC code into a shared library is impossible on x86_64, but is allowed on ix86 (and leads to many subtle problems, like this one).

Update:

I have 1 module compiled without -fPIC, but I do not set tls-model at all, as far as I remember the default value is not initial-exec

• There could only be one tls model for each ELF image (executable or shared library).
• TLS model defaults to initial-exec for non-fPIC code.

It follows that if you link even one non-fPIC object that uses __thread into foo.so, then foo.so gets initial-exec for all of its TLS.

So why it causes problems - because if initial-exec is used then the number of tls variables is limited (because they are not dynamically allocated)?

Correct.