c++linuxgccdlopen

How to determine the maximum static TLS (Thread Local Storage) Block Size on Linux?


Problem

Trying to run this code

if (!dlopen("../lib/libMy.so", RTLD_NOW)) {
    std::cout << dlerror() << std::endl;
    return 1;
}

resulted in this TLS (Thread Local Storage) error:

cannot allocate memory in static TLS block

Solution

Dumping the Thread Local Storage section .tbss (static and non static non-initialized thread local variables) shows Tls_ErrorStruct size is 600 bytes (0x258) and Tls_tv is 1600 bytes (0x640), totaling 2200 TLS bytes:

$ objdump -C -t ../lib/libMy.so | grep -F '.tbss'

00000000000008e8 l       .tbss  0000000000000008              runtime.tlsg
0000000000000010 l       .tbss  0000000000000001              __tls_guard
0000000000000688 l       .tbss  0000000000000001              __tls_guard
...
0000000000000690 g       .tbss  0000000000000258              Tls_ErrorStruct
...
0000000000000018 g       .tbss  0000000000000640              Tls_tv

To fix the block error, those thread_local variables were reduced to ~600 bytes. The dlopen now succeeds.

Note

To check how much Thread Local Storage (TLS) is needed 0x0008f0

$ readelf -Wl ../lib/libMy.so

Elf file type is DYN (Shared object file)
Entry point 0x0
There are 10 program headers, starting at offset 64

Program Headers:
  Type           Offset    VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  LOAD           0x000000  0x0000000000000000 0x0000000000000000 0x48ada8 0x48ada8 R   0x1000
  LOAD           0x48b000  0x000000000048b000 0x000000000048b000 0xdab271 0xdab271 R E 0x1000
  LOAD           0x1237000 0x0000000001237000 0x0000000001237000 0x2c0564 0x2c0564 R   0x1000
  LOAD           0x14f76a0 0x00000000014f86a0 0x00000000014f86a0 0xd16172 0xd6ec50 RW  0x1000
  DYNAMIC        0x20b58c0 0x00000000020b68c0 0x00000000020b68c0 0x000220 0x000220 RW  0x8
  NOTE           0x000270  0x0000000000000270 0x0000000000000270 0x000088 0x000088 R   0x4
  TLS            0x14f76a0 0x00000000014f86a0 0x00000000014f86a0 0x000001 0x0008f0 R   0x8
  GNU_EH_FRAME   0x13fa440 0x00000000013fa440 0x00000000013fa440 0x02358c 0x02358c R   0x4
  GNU_STACK      0x000000  0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x10
  GNU_RELRO      0x14f76a0 0x00000000014f86a0 0x00000000014f86a0 0xbc3960 0xbc3960 R   0x1

Source

Question

I'm not sure how to find the dynamic loader Thread Local Storage block size but did find a possible reference here:

/* Size of the static TLS block.  Giving this initialized value
   preallocates some surplus bytes in the static TLS area.  */
size_t _dl_tls_static_size = 2048;

Is there a linux tool or some C++ method to determine the max Thread Local Storage block size?


Solution

  • The reason libTestTLS.so loads successfully in this answer is that the tread-local variable is never used.

    gdb -q ./LoadTLS
    
    (gdb) start
    Temporary breakpoint 1 at 0x40117f: file LoadTLS.c, line 5.
    Starting program: /tmp/tls/LoadTLS
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib64/libthread_db.so.1".
    
    Temporary breakpoint 1, main () at LoadTLS.c:5
    5           void* handle = dlopen("./libTestTLS.so", RTLD_NOW);
    (gdb) n
    6           if (!handle) {
    (gdb) p handle
    $1 = (void *) 0x4172d0
    (gdb) p tv
    The inferior has not yet allocated storage for thread-local variables in
    the shared library `./libTestTLS.so'
    for Thread 0x7ffff7a7d240 (LWP 176)
    

    If you add an actual use for it:

    class ThreadVector
    {
    public:
        char tlsBlock[1 * 1024 * 1024];
    };
    
    thread_local ThreadVector tv;
    
    #if defined(FN)
    char* fn()
    {
      return tv.tlsBlock;
    }
    #endif
    

    and rebuild the library with g++ -g -fPIC -shared -o libTestTLS.so TestTLS.c -ftls-model=initial-exec -DFN, then you can see the failure:

    ./LoadTLS
    Failed to load library: ./libTestTLS.so: cannot allocate memory in static TLS block