Convince Python-CFFI to find shared lib

I have a somewhat strange setup: I have a libmylib.so I cannot modify, but, since the interface is rather convoluted, I wrote a simple wrapper directly into ffbuilder.set_source()

My full lib_builder.py is:

import os
from cffi import FFI
ffibuilder = FFI()

ffibuilder.cdef("""
int start(int port);
int ready(void);
int publish(char *buf, int len);
int stop(void);
""")

ffibuilder.set_source('_mylib_cffi', """
#include "mylib.h"

static uint32_t handle = 0xffffffff;
int start(int port) {
    mylib_init(0);
    mylib_register_publisher("127.0.0.1", port, &handle, 0);
    return 1;
}
int ready(void) {
    return handle != 0xffffffff;
}
int publish(char *buf, int len) {
    return mylib_publish(handle, buf, len);
}
int stop(void) {
    mylib_shutdown();
    return 1;
}
""", libraries=['mylib'], library_dirs=[os.path.dirname(__file__)])

if __name__ == '__main__':
    ffibuilder.compile(verbose=True)

This works as expected but my test code:

import _mylib_cffi
...

bombs because libmylib.so is not found (it is in current directory, exactly where generated _mylib_cffi.cpython-310-x86_64-linux-gnu.so is located).

I can make it work either moving libmylib.so to /usr/lib (or other directory in the system lib search path) or adding current directory to LD_LIBRARY_PATH.

Both solutions are not particularly appealing to me because I don't want to pollute system settings with needs of a single program.

Is there some way to dynamically load needed lib from within Python?

Note: I'm trying to learn Python-CFFI, so any advice is welcome.

Solution

At least with GCC on Linux, you can avoid the LD_LIBRARY_PATH hack (described in your own answer to the question) with the linker's -rpath option. I think what you need is:

ffibuilder.set_source(..., extra_link_args=['-Wl,-rpath=$ORIGIN'])

I didn't test it, but I think this should make CFFI produce the glue .so in such a way that this .so should look for libmylib.so in the same directory as where it is.

On Windows, I think nothing is needed: a DLL always (by default) tries to look for other DLLs it depends on in the same directory.