Both main program and dlopen'ed library need different versions of libsqlite3.so

I have a main program that depends on libsqlite3.so. By running it with LD_DEBUG=libs ./my_executable, I observe the following (I edited the search paths to look simpler):

    386600:     find library=libsqlite3.so.0 [0]; searching
    386600:      search path=/a:/b:/c:/d:/e              (RPATH from file /home/me/my_executable)
    386600:       trying file=/a/libsqlite3.so.0
    386600:       trying file=/b/libsqlite3.so.0
    386600:       trying file=/c/libsqlite3.so.0
    386600:       trying file=/d/libsqlite3.so.0
    386600:       trying file=/e/libsqlite3.so.0

So far so good, the runtime linker correctly uses the RPATH from my_executable to eventually find libsqlite3.so in /e.

Later on, my_executable goes on to run a python interpreter. Eventually, a python import call tries to load libsqlite3 and it gets resolved according to the RPATH from my_executable, and ldd throws an error about unresolved symbols.

Thankfully, in this specific case, I can just run LD_DEBUG=libs python -c "import IPython" to see what Python wants:

    396818:     find library=libsqlite3.so.0 [0]; searching
    396818:      search path=/home/me/miniconda3/lib/python3.9/lib-dynload/../..             (RPATH from file /home/me/miniconda3/lib/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so)
    396818:       trying file=/home/me/miniconda3/lib/python3.9/lib-dynload/../../libsqlite3.so.0

So, my_executable and python want different versions of libsqlite3. First of all, is it possible to dlopen a second version of a shared library at runtime, and how do I tell dlopen to ignore the RPATH from my_executable?

Solution

First of all, is it possible to dlopen a second version of a shared library at runtime

Yes, so long as the two libraries differ in their SONAME (which they probably don't, but you can binary-patch one or both).

However, doing that is really bad idea -- the two libraries are exceedingly likely to define the same set of symbols, and the linux rules are that the first definition wins.

Doing this will probably cause very hard to debug crashes.

Effectively what's happened here is that you have two libraries with the same SONAME, but different ABI (we know the ABI's are different because if they weren't, you would not have gotten unresolved symbols -- it would all have just worked).

and how do I tell dlopen to ignore the RPATH from my_executable

You can't. But you can dlopen() the other libsqlite3.so.0 using absolute path (provided it has different SONAME).

Your best bet is probably to do what n.m. suggested: use dlmopen in your main program to load your version of libsqlite3.so.0 into a separate loader namespace.