I have compiled C code to be called by a Python script. Of course I can include it with cdll.LoadLibrary("./whatever.so"), but I would prefer it to be accessible to all Python scripts in different folders. The idea is that I use default paths for shared objects and do not change environment variables or system files to do that.
According to one of the answers on Why can't Python find shared objects that are in directories in sys.path?, /usr/local/lib should work. Namely, /etc/ld.so.conf.d/libc.conf includes that folder. So I used sudo cp -a whatever.so /usr/local/lib and sudo ldconfig. However, cdll.LoadLibrary("whatever.so") does not find the file.
Following other suggestions, I have run python -m site, and /usr/local/lib is unfortunately not on the list. Probably the third element, /usr/lib/python3.9, is the best choice, but how can I automatically select it on the cp command?
To summarise, is there a good default place to put shared objects (.so) without having to change environment variables and/or system files, and how can I choose it automatically. [I want to write a such Makefile code that puts compiled shared object into path.]
If you do things right, then a shared library libfoo.so[.x.y.z] placed in /usr/local/lib (or in any of the directories listed in
files /etc/ld.so.conf.d/*.conf) will be found in python3 by:
cdll.LoadLibrary('libfoo.so.[x.y.z]').
For a shared library of your own making, /usr/local/lib is the appropriate place. For example:
$ cat foo.c
int foo(void) { return 42; }
$ gcc -shared -o libfoo.so foo.c
$ sudo cp libfoo.so /usr/local/lib/
$ sudo ldconfig
$ python3
Python 3.12.3 (main, Nov 6 2024, 18:32:19) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import cdll
>>> cdll.LoadLibrary('libfoo.so')
<CDLL 'libfoo.so', handle 2dbdb10 at 0x769975b3a930>
>>>
works, and for an existing system library:
$ find /usr/lib/ -name libgmp.so.*
/usr/lib/x86_64-linux-gnu/libgmp.so.10.5.0
/usr/lib/x86_64-linux-gnu/libgmp.so.10
$ python3
Python 3.12.3 (main, Nov 6 2024, 18:32:19) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import cdll
>>> cdll.LoadLibrary('libgmp.so.10')
<CDLL 'libgmp.so.10', handle aaae150 at 0x70b0c6938590>
>>>
works.
I surmise that your mistake must lie in the actual name that you have given to your whatever.so.
You cannot call a shared library whatever you like if it is to be found by the dynamic linker
(or the static linker) in their usual ways. A shared library needs to have a name of the form
lib<name>.so[.x.y.z]. If not, one of the snags that result is that ldconfig will ignore it.
Then the dynamic linker will not find it, hence neither will cdll.LoadLibrary. As you can see
if we try to repeat the libfoo.so experiment after removing lib from its name:
$ mv libfoo.so foo.so
$ sudo cp foo.so /usr/local/lib
$ sudo ldconfig
$ sudo ldconfig --print-cache | grep foo.so
libfoo.so (libc6,x86-64) => /usr/local/lib/libfoo.so
libfoo.so is still in the ldconfig cache since the first experiment, but foo.so
has not been added, and the dynamic linker can't find it:
$ python3
Python 3.12.3 (main, Nov 6 2024, 18:32:19) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import cdll
>>> cdll.LoadLibrary('foo.so')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.12/ctypes/__init__.py", line 460, in LoadLibrary
return self._dlltype(name)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/ctypes/__init__.py", line 379, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: foo.so: cannot open shared object file: No such file or directory
>>>
The requirement for shared library names to begin with lib is documented in man ldconfig
ldconfig will look only at files that are named lib*.so* (for regular shared objects) or ld-.so (for the dynamic loader itself). Other files will be ignored.