Trying to use my native library "contribtests.so" which accesses the dynamic compiled BZip2 library "libbz2.so" always fails when invoking System.loadLibrary("contribtests")
java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "BZ2_bzlibVersion" referenced by "/data/app/~~aTgFU-pNoDjbLJvJTuDiiQ==/com.microcaet.contribtests
I unpacked the generated apk and use readelf to examine the sitation. Both libraries are seem to be perfectly well
[llothar@linux-workstation a]$ readelf -d lib/arm64-v8a/libbz2.so
Dynamic section at offset 0xf478 contains 24 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libdl.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so]
0x000000000000000e (SONAME) Library soname: [libbz2.so]
0x000000000000001e (FLAGS) BIND_NOW
0x000000006ffffffb (FLAGS_1) Flags: NOW
0x0000000000000007 (RELA) 0xd40
0x0000000000000008 (RELASZ) 192 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffff9 (RELACOUNT) 3
0x0000000000000017 (JMPREL) 0xe00
0x0000000000000002 (PLTRELSZ) 840 (bytes)
0x0000000000000003 (PLTGOT) 0x11620
0x0000000000000014 (PLTREL) RELA
0x0000000000000006 (SYMTAB) 0x308
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000005 (STRTAB) 0x20098
0x000000000000000a (STRSZ) 800 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x928
0x000000000000001a (FINI_ARRAY) 0x11468
0x000000000000001c (FINI_ARRAYSZ) 16 (bytes)
0x000000006ffffff0 (VERSYM) 0x890
0x000000006ffffffe (VERNEED) 0x908
0x000000006fffffff (VERNEEDNUM) 1
0x0000000000000000 (NULL) 0x0
and this is the one containing JNI calls which call into bz2
[llothar@linux-workstation a]$ readelf -d lib/arm64-v8a/libcontribtests.so
Dynamic section at offset 0x144d18 contains 41 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libzip.so]
0x0000000000000001 (NEEDED) Shared library: [libcurl.so]
0x0000000000000001 (NEEDED) Shared library: [libnghttp2.so]
0x0000000000000001 (NEEDED) Shared library: [libssl.so]
0x0000000000000001 (NEEDED) Shared library: [libcrypto.so]
0x0000000000000001 (NEEDED) Shared library: [libarchive.so]
0x0000000000000001 (NEEDED) Shared library: [libcares.so]
0x0000000000000001 (NEEDED) Shared library: [liblzma.so]
0x0000000000000001 (NEEDED) Shared library: [libssh.so]
0x0000000000000001 (NEEDED) Shared library: [libbz2.so]
0x0000000000000001 (NEEDED) Shared library: [libzstd.so]
0x0000000000000001 (NEEDED) Shared library: [libz.so]
0x0000000000000001 (NEEDED) Shared library: [libandroid.so]
0x0000000000000001 (NEEDED) Shared library: [liblog.so]
0x0000000000000001 (NEEDED) Shared library: [libm.so]
0x0000000000000001 (NEEDED) Shared library: [libdl.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so]
0x000000000000000e (SONAME) Library soname: [libcontribtests.so]
0x000000000000001e (FLAGS) BIND_NOW
0x000000006ffffffb (FLAGS_1) Flags: NOW
0x0000000000000007 (RELA) 0x43268
0x0000000000000008 (RELASZ) 78840 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffff9 (RELACOUNT) 1287
0x0000000000000017 (JMPREL) 0x56660
0x0000000000000002 (PLTRELSZ) 27792 (bytes)
0x0000000000000003 (PLTGOT) 0x147588
0x0000000000000014 (PLTREL) RELA
0x0000000000000006 (SYMTAB) 0x330
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000005 (STRTAB) 0x188d4
0x000000000000000a (STRSZ) 174480 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x13188
0x0000000000000019 (INIT_ARRAY) 0x146ce8
0x000000000000001b (INIT_ARRAYSZ) 48 (bytes)
0x000000000000001a (FINI_ARRAY) 0x146cd8
0x000000000000001c (FINI_ARRAYSZ) 16 (bytes)
0x000000006ffffff0 (VERSYM) 0x119e8
0x000000006ffffffe (VERNEED) 0x13124
0x000000006fffffff (VERNEEDNUM) 3
0x0000000000000000 (NULL) 0x0
When looking for the mentioned function name in the error message i found the symbols defined in the libbz2.so and marked as UND (undefined) in the calling library.
[llothar@linux-workstation a]$ readelf -s lib/arm64-v8a/libbz2.so | grep bzlibVer
40: 000000000000d34c 12 FUNC GLOBAL DEFAULT 10 BZ2_bzlibVersion
[llothar@linux-workstation a]$ readelf -s lib/arm64-v8a/libcontribtests.so | grep bzlibVer
5: 0000000000000000 0 FUNC GLOBAL DEFAULT UND BZ2_bzlibVersion
[llothar@linux-workstation a]$
The only idea is that it might happen because the compilation creates a library with semantic versioning numbres "libbz2.so.1.0.9". I'm using patchelf tool to remove the ".1.0.9" doing "patchelf --set-soname new_name old_name" and then renameing the file because otherwise gradle would not pack the version into the resulting apk.
As a test, i also generated "contribtests" as a simple "main.c" to call the function and executed it as command line program (uploaded via adb). This is working with exactly the same renamed "libbz2.so" that i copy into the JniLib location. But i set LD_LIBRARY_PATH before running it.
How can i find what is going on? I'm totally helpless right now. How to fix or debug this problems? I assume the so files are correctly loaded already. So something in the dark deepth of symbol solving is creating this problem.
Ok, i found the problem.
While "patchelf --set-soname newName oldName" is removing the numbering in the SO name, the linker will set also some other hidden places in the executable that patchelf wont update.
The only way to make it link is to patch the build scripts and not create symbolic version in the SO files in the first place. This is bad as there is no way to configure a common build script this without patching (unless i missed something). Makes porting of existing code not so easy.
Just try to find the "-Wl," line or in cmake the "set_target_property" and strip the internal numbers. This will still generate the numbered filename and symlinks which have to get removed and renamed afterwards via a shell script.