c++linuxopencvstatic-librariesstatic-linking

Is it possible to build opencv statically under Linux without having problems with missing/incompatible libraries?

I'm trying to distribute an opencv binary on different Linux operating systems, but I've failed because of dependency problems. What I'm aiming for is something similar to the opencv-python pip package that doesn't require additional software to be installed.

My questions are as follows:

OpenCV were built statically.

I first had a problem with glibc version incompatibility. An update on the different systems solved the problem.

Then I had a missing libtiff.so error. set(BUILD_TIFF ON) solved the problem.

Then I got an error that libQtWidgets.so was missing. I installed the qtbase5-dev package. However, I don't want users to have to install new software. I tried to build qtbase statically, but it complained that the opengl library was missing. This means that the static library will still require opengl to be installed on the destination machines. Which again is incompatible with my goal.

Even after installing qtbase5-dev, I had another version incompatibility with libavcodec.

Update 1

Looking at cv2.abi3.so gave more information.

readelf -d cv2.abi3.so shows

 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../opencv_python.libs]
 0x0000000000000001 (NEEDED)             Shared library: [libpng16-7379b3c3.so.16.40.0]
 0x0000000000000001 (NEEDED)             Shared library: [libavcodec-512f0acb.so.59.37.100]
 0x0000000000000001 (NEEDED)             Shared library: [libavformat-3ff1be5b.so.59.27.100]
 0x0000000000000001 (NEEDED)             Shared library: [libavutil-a0a0531e.so.57.28.100]
 0x0000000000000001 (NEEDED)             Shared library: [libswscale-2c3c8be7.so.6.7.100]
 0x0000000000000001 (NEEDED)             Shared library: [libQt5Widgets-e69d94fb.so.5.15.0]
 0x0000000000000001 (NEEDED)             Shared library: [libQt5Gui-a7aedf18.so.5.15.0]
 0x0000000000000001 (NEEDED)             Shared library: [libQt5Test-c38a5234.so.5.15.0]
 0x0000000000000001 (NEEDED)             Shared library: [libQt5Core-39545cc7.so.5.15.0]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libopenblas-r0-f650aae0.3.3.so]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]

Some of the dependencies names have been rewritten and shipped with the package in the opencv_python.libs directory. I intend to do the same.

The renaming process seems to be part of the packaging process. Numpy has similiar renamings, making me think that the renaming process is outside python-opencv

I am still looking for that renaming process

Solution

  • I found a solution.

    opencv-python uses auditwheel to relabel cross-distribution Linux wheels.

    There is a workaround to relabel any shared libraries.

    If maximum portability is needed, use some of the manylinux images as a build hosts. I have used the image opencv-python-manylinux2014-x86-64.

    Install python and needed packages

    python -m pip install build auditwheel

    [Optional] Exclude shared libraries we don't want to relabel

    Execute the follwing pyton script

    from os.path import join, dirname, abspath
import json

from auditwheel import policy

def patch_whitelisted_libs():
    policies = None

    with open(join(dirname(abspath(policy.__file__)), "manylinux-policy.json")) as f:
        policies = json.load(f)

    lib_whitelist = [
        "libxcb.so.1",
    ]

    lib_blacklist = []

    changed = False

    for p in policies:
        for lib in lib_whitelist:
            if not (lib in p["lib_whitelist"]):
                p["lib_whitelist"].append(lib)
                changed = True

        for lib in lib_blacklist:
            if lib in p["lib_whitelist"]:
                p["lib_whitelist"].remove(lib)
                changed = True

    if changed:
        with open(join(dirname(abspath(policy.__file__)), "manylinux-policy.json"), "w") as f:
            f.write(json.dumps(policies))

if __name__ == '__main__':
    patch_whitelisted_libs()

    Create a python project specifying your distributed shared libraries

    setup.py

    from setuptools import setup, Extension, find_packages

ext_module = Extension('package_name', sources = [], libraries = [])

setup(
    name='package_name',
    version = '0.0.0',
    packages = ['package_name'],
    package_dir = {'package_name': '/path/to/shared_objects/directory'},
    package_data = {'package_name': ['opencv_custom.so', 'qt/plugins/platforms/libqxcb.so']},
    ext_modules = [ext_module],
)

    pyproject.toml

    [build-system]
requires = [
    "setuptools>=58"
]

build-backend = "setuptools.build_meta"

    Build the python project

    python -m build --wheel

    Relabel the shared objects

    python -m auditwheel repair dist/<generated built wheel>

    Extract the relabelled shared objects

    unzip -o -d relabelled dist/<generated built wheel> 'package_name/*' 'package_name.libs/*'

    [Optional] change rpath to suit your distribution tree layout

    auditwhell expects you to keep the tree layout extracted. If it is not the case, change the rpath of every shared object defined by package_data in the setup.py file with

    patchelf --force-rpath --set-rpath '$ORIGIN/<you tree layout>' <your shared object>

    QT plugins and fonts problem

    QT may complain about not being able to load a plugin. I solved it by setting an environment variable

    QT_QPA_PLATFORM_PLUGIN_PATH=/path/to/the/distributed/plugins

    QT may complain about not founding fonts. I solved it by setting an environment variable

    QT_QPA_FONTDIR=/path/to/the/distributed/fonts/dejavu