Example: Java source Foo.java:
public class Foo {
public static void main(String[] args) {}
};
I run it on Ubuntu 24.04 docker with address sanitizer (contains leak sanitizer) preloaded, plus an interceptor for dlopen/dlclose according to github issue. Java compiler & launcher are from openjdk-8-jdk package.
javac Foo.java
LD_PRELOAD="/usr/lib/llvm-17/lib/clang/17/lib/linux/libclang_rt.asan-x86_64.so:/root/libinterceptor.so" java Foo
The interceptor code is
#define _GNU_SOURCE
#include <dlfcn.h>
#include <link.h>
#include <stdio.h>
#include <string.h>
int dlclose(void *handle) {
printf("Intercepted a dlclose call, handle %ld\n", (intptr_t)handle);
return 0;
}
void* dlopen(const char* filename, int flags){
typedef void* (*dlopen_t)(const char*, int);
dlopen_t original_dlopen = (dlopen_t)dlsym(RTLD_NEXT, "dlopen");
flags |= RTLD_NODELETE;
void *ret = original_dlopen(filename, flags);
printf("Intercepted a dlopen call for %s, handle %ld, injecting RTLD_NODELETE\n", filename, (intptr_t)ret);
return ret;
}
Many (false positive) memory leaks are reported. This is fine, they're most likely due to garbage collection and inability of leak sanitizer to understand what's happening there. I'm not asking what to do with them, I know I should ignore them as they're false positives, I'm using them to demonstrate the issue at hand. Part of the lsan error output:
Direct leak of 120 byte(s) in 1 object(s) allocated from:
#0 0x7f3d1c0fb372 in malloc (/usr/lib/llvm-17/lib/clang/17/lib/linux/libclang_rt.asan-x86_64.so+0xfb372) (BuildId: 91f375f2a48c6b133a56d8cc059d017ae5de4982)
#1 0x7f3d17f8c7b4 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x98c7b4) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#2 0x7f3d178d29bd (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x2d29bd) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#3 0x7f3d178d2a7a (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x2d2a7a) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#4 0x7f3d17af07a9 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x4f07a9) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#5 0x7f3d17a5ac4d (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x45ac4d) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#6 0x7f3d17a5c3ca (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x45c3ca) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#7 0x7f3d17a6323a (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x46323a) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#8 0x7f3d180c3a34 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0xac3a34) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#9 0x7f3d180c4680 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0xac4680) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#10 0x7f3d180c64c4 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0xac64c4) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#11 0x7f3d17af25d7 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x4f25d7) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#12 0x7f3d17af36b9 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x4f36b9) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#13 0x7f3d17e59707 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x859707) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#14 0x7f3d17e5b308 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x85b308) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#15 0x7f3d17ca99cb (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x6a99cb) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#16 0x7f3d07a24dc7 (<unknown module>)
#17 0x7f3d07a080f5 (<unknown module>)
#18 0x7f3d07a004e6 (<unknown module>)
#19 0x7f3d17cb2884 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x6b2884) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#20 0x7f3d17d2e75e (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x72e75e) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#21 0x7f3d17d2efbf (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x72efbf) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#22 0x7f3d1bac6c8a in JNU_NewStringPlatform (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libjava.so+0x1dc8a) (BuildId: 95ab745b31a25f66a623ffeaf6822cdc641c5fab)
#23 0x7f3d1babf2cb in Java_java_lang_System_initProperties (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libjava.so+0x162cb) (BuildId: 95ab745b31a25f66a623ffeaf6822cdc641c5fab)
#24 0x7f3d07a185e6 (<unknown module>)
#25 0x7f3d07a07e3f (<unknown module>)
#26 0x7f3d07a004e6 (<unknown module>)
#27 0x7f3d17cb2884 (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x6b2884) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#28 0x7f3d17cb115e (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x6b115e) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
#29 0x7f3d17cb179f (/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so+0x6b179f) (BuildId: d38655f827ac6a65fdbe1898b1278717fdb89a4e)
Here's the issue: quite a lot of those leak reports have <unknown module>s in stack trace. My question is how to turn them into known libraries so that I could trace what happens. I'm not really asking specifically about Java, this is just an example where I encountered it - but it might turn out specific to Java launcher or other programs.
If the <unknown module>s were caused by dynamic loading or unloading, then intercepting dlopen or dlclose should be sufficient. However, the only standard output I'm getting is
Intercepted a dlopen call for /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so, handle 89747636617344, injecting RTLD_NODELETE
Intercepted a dlopen call for librt.so.1, handle 89747636681856, injecting RTLD_NODELETE
Intercepted a dlopen call for /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libverify.so, handle 89747636684928, injecting RTLD_NODELETE
Intercepted a dlopen call for /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libjava.so, handle 89747636686464, injecting RTLD_NODELETE
Intercepted a dlopen call for /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libzip.so, handle 89747636688000, injecting RTLD_NODELETE
Intercepted a dlopen call for (null), handle 139900452795104, injecting RTLD_NODELETE
Intercepted a dlopen call for /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libzip.so, handle 89747636688000, injecting RTLD_NODELETE
Intercepted a dlopen call for /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libnio.so, handle 89747636689536, injecting RTLD_NODELETE
There are no dlclose lines and none of the handles (converted to hex, not counting main program's for nullptr filename) match the range of addresses that appear in <unknown module>s - perhaps something to do with ASLR. Most importantly, <unknown module>s still appear in lsan error output, so either this approach is unable to prevent unloading, or the issue isn't caused by dynamic loading/unloading.
How can I find out which libraries correspond to <unknown module>s?
The JVM JIT-compiled code and certain internal applications might be showing up as <unknown module>, so the easiest way to confirm this is to map the raw addresses against actual loaded libraries. The typical approach is to check /proc/[pid]/maps (or use something like pmap) during runtime and see which regions match up with those reported addresses.
You could also try enabling more verbose ASan/LSan options:
export ASAN_SYMBOLIZER_PATH to point to LLVM-symbolizer and set ASAN_OPTIONS=symbolize=1,print_module_map=2 (and similarly for LSAN_OPTIONS) so the sanitizer spits out more detailed library mappings that might otherwise be missed. If it still shows <unknown module>, it's likely JIT code or something the JVM allocated in a private mapping that doesn't have a corresponding file-backed library. Sometimes you can force the JVM to preserve more frame pointers or generate more debuggable code (using -XX:+PreserveFramePointer and -XX:+UnlockDiagnosticsVMOptions -XX:+DebugNonSafepoints so the sanitizer can unwind and label those stack frames better.