I try to create a self contained wheel for pyRFC: http://sap.github.io/PyRFC/install.html
I have the needed (closed sourced) libraries and header files.
Installing pyRFC works, if I copy the libraries and headers to $VIRTUAL_ENV/lib and $VIRTUAL_ENV/include.
Now I try to create a wheel which contains the closed source library files.
But I am failing.
If I unzip the wheel it only contains this:
(pypi)pypi@pypiserver:~> unzip -l packages/pyrfc-1.9.91-cp27-cp27mu-linux_x86_64.whl
Archive: packages/pyrfc-1.9.91-cp27-cp27mu-linux_x86_64.whl
Length Date Time Name
--------- ---------- ----- ----
5366 2018-07-23 13:38 pyrfc/_exception.py
1045 2018-07-23 13:38 pyrfc/__init__.py
1610216 2018-07-23 14:10 pyrfc/_pyrfc.so
3835 2018-07-23 14:10 pyrfc-1.9.91.dist-info/DESCRIPTION.rst
990 2018-07-23 14:10 pyrfc-1.9.91.dist-info/metadata.json
6 2018-07-23 14:10 pyrfc-1.9.91.dist-info/top_level.txt
105 2018-07-23 14:10 pyrfc-1.9.91.dist-info/WHEEL
4666 2018-07-23 14:10 pyrfc-1.9.91.dist-info/METADATA
715 2018-07-23 14:10 pyrfc-1.9.91.dist-info/RECORD
--------- -------
1626944 9 files
The extra_objects (see below) are missing.
How can I modify the setup.py of pyRFC to make the wheel contain the libraries from $VIRTUAL_ENV/lib?
Here is the setup.py: https://github.com/SAP/PyRFC/blob/master/setup.py
I tried this patch for setup.py
@@ -48,7 +49,9 @@ PYRFC_EXT = Extension(
, libraries=LIBS
, define_macros=MACROS
, extra_compile_args=COMPILE_ARGS
- , extra_link_args=LINK_ARGS
+ , extra_link_args=LINK_ARGS,
+ library_dirs=['lib'],
+ extra_objects = ['lib/libicudata.so.50', 'lib/libsapnwrfc.so', 'lib/libicui18n.so.50', 'lib/libicuuc.so.50', 'lib/libicudecnumber.so', 'lib/libsapucum.so'],
)
If I install and run the library without libsapnwrfc.so I get this error:
Traceback (most recent call last):
File "test-pyrfc.py", line 1, in <module>
from pyrfc import Connection
File "/home/other/lib/python2.7/site-packages/pyrfc/__init__.py", line 22, in <module>
from pyrfc._pyrfc import get_nwrfclib_version, Connection, TypeDescription, FunctionDescription, Server
ImportError: libsapnwrfc.so: cannot open shared object file: No such file or directory
If I trace the open calls, I see that it only looks at root-level for the library. The library does not get search in the virtualenv (which is /home/other):
strace python test-pyrfc.py 2>&1 | grep libsapnwrfc.so
open("/lib64/tls/x86_64/libsapnwrfc.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib64/tls/libsapnwrfc.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib64/x86_64/libsapnwrfc.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib64/libsapnwrfc.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/x86_64/libsapnwrfc.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/libsapnwrfc.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/x86_64/libsapnwrfc.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/libsapnwrfc.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "libsapnwrfc.so: cannot open shar"..., 73libsapnwrfc.so: cannot open shared object file: No such file or directory) = 73
I would like to install "libsapnwrfc.so" via wheel into a virtualenv because this gives me the power to have several independent environments. I could install it via RPM or config-management, but I would like to avoid it.
The extra_objects are used only when linking the extension lib and are not bundled in the wheel. From the docs:
extra_objects: list of extra files to link with (eg. object files not implied by ‘sources’, static library that must be explicitly specified, binary resource files, etc.)
For bundling the libs into wheel, use auditwheel for Linux wheels, or delocate for MacOS wheels. SO already has an excellent question: How to build and distribute a Python/Cython package that depends on third party libFoo.so which covers the whole wheel repairing process using both auditwheel and delocate. Here, I will only summarize the necessary commands. The process is pretty similar with both tools:
$ pip install auditwheel # you may also need to install patchelf
$ python setup.py bdist_wheel
$ auditwheel show dist/*_linux_x86_64.whl # will show the libs to be bundled
$ auditwheel repair dist/*_linux_x86_64.whl
This will create a new dir dist/wheelhouse, containing the new wheel with the bundled libs.