[SOLVED] Memory release in pthread TLS destructor is not detected by valgrind/massif

Memory release in pthread TLS destructor is not detected by valgrind/massif

Symptoms: I allocate TLS key with a destructor, create a bundle of threads and pass the TLS key to each thread. Each thread allocates memory and sets its pointer in TLS, the TLS destructor deallocates memory. I wait for threads to finish before app exits. The app is run under valgrind/massif that reports this memory not deallocated.

int main(int argc, char **argv)
{
  pthread_key_t* key = new pthread_key_t();
  pthread_key_create(key, my_destructor);

  pthread_t threads[32000];

  for(int i=0; i<32000; ++i)
    pthread_create(&threads[i], NULL, my_thread, key);

  for(int i=0; i<32000; ++i)
    pthread_join(threads[i], NULL);

  return 0;
}

In the thread runner I allocate the memory and set it up in the TLS:

extern "C" void* my_thread(void* p)
{
  pthread_setspecific(*(pthread_key_t*)p, malloc(100));

  return NULL;
}

In the TLS destructor, I release the memory:

extern "C" void my_destructor(void *p)
{
  free(p);
}

I run this under valgrind/massif 3.19 with the following options:

  --tool=massif
  --heap=yes
  --pages-as-heap=yes
  --log-file=/tmp/my.log
  --massif-out-file=/tmp/my.massif.log

Then I run ms_print /tmp/my.massif.log. I am getting the leaks reported like the following:

| ->01.75% (67,108,864B) 0x76F92D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->01.75% (67,108,864B) 0x76F98D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| |   ->01.75% (67,108,864B) 0x76FF77D: malloc (in /usr/lib64/libc-2.17.so)
| |     ->01.75% (67,108,864B) 0x410300: my_thread (threadsT.cpp:136)
| |       ...
| |       <skipped by author>
| |       ...
| |             
| ->00.00% (73,728B) in 1+ places, all below ms_print's threshold (01.00%)

...while I would not expect anything reported leaked at all.

I added the instrumentation to my_destructor and manually verified that:

it is invoked, indeed
it deallocates the memory, as it is supposed to do

Is there something apparent I am doing wrong here that makes valgrind/massif report these? Is it a valgrind/massif limitation that it cannot detect the memory deallocation when invoked from TLS destructors?

Building and running that with gcc 4.9.4 on Red Hat Enterprise Linux Server release 7.9 (Maipo).

Solution

A second answer, this time concentrating on the 'leak' aspect.

Massif isn't really a leak detector. It's for profiling heap use.

If I compile the example (with 320 threads) then at the end I get about 89 million bytes still allocated. That is made up of

75% the arena used by malloc called from start_thread
9% pthread_create
15% loading shared libraries

None of that looks like much of a concern to me. I assume that the start_thread memory is the pthread stack cache.

If I use massif for profiling malloc/new, then the last sample is

  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
73      2,929,610            2,360            2,308            52            0