We are trying to track down a memory leak in proxysql (2.0.14) which uses jemalloc (5.2.0). We compiled proxysql with debug symbols enabled. The jemalloc configuration that is baked into proxysql is the following:
xmalloc:true,lg_tcache_max:16,prof:true,prof_leak:true,lg_prof_sample:20,lg_prof_interval:30,prof_active:false
The memory profiling is enabled at runtime with PROXYSQL MEMPROFILE START as can be seen here. This sets memory profiling to active:
bool en=true;
mallctl("prof.active", NULL, NULL, &en, sizeof(bool));
This appears to work as the heap profile files are being generated, however when we analyse them with jeprof, it appears to log all memory allocations and not just the memory leaks. The pdf that we generated using jeprof can be found here. As you can see, it seems like jeprof shows all of the memory allocations, instead of just the leaked ones.
From looking at other memory profiles, they display a smaller amount of memory total, whereas here in this profile we are seeing 21GB of total memory, which is more than what the server could allocate. However, we are not sure where the problem lies - with jeprof, the configuration, or something else entirely.
For reference we have added a sample heap profile here in case this will give an indication of what is going wrong.
I think this may reflect some of the ambiguity in what "leak" means. Tools like Valgrind will traverse the pointer graph of the program at shutdown, and treat things as leaks if that traversal does not find some live allocation.
Other tools treat "leaks" as simply anything allocated but not freed. The profiling mode in jemalloc works more like this way. It reports an estimation of the live bytes attributable to various call stacks at program termination.