linuxqtlinkerldqmake

Why is necessary to provide path to the library if ldconfig is used?

I have Qt[4|5] project with QMake build configuration and several library dependencies:

  1. OpenCV library installed by zypper
  2. Custom libnf_cln.so library installed in /some/path

I created /etc/ld.so.conf.d/nf_cln.conf with /some/path as a content. I run sudo ldconfig.

Project succeed to build with follow code:

LIBS += -L/some/path
LIBS += -lopencv_core -lnf_cln

It fails if I remove follow line:

LIBS += -L/some/path

Why it is necessary to point location for nf_cln and isn't necessary to point it for opencv_core?

Solution

  • Why it is necessary to point location for nf_cln and isn't necessary to point it for opencv_core

    Because libopencv_core is installed in one of the static linker's default search directories and the custom shared library libnf_cln is not: it's in /some/path.

    The ldconfig cache, controlled through the files in /etc/ld.so.conf.d/ is - perhaps confusingly - not used by the static linker, ld, to find libraries that are input on the linkage commandline to make a program or shared library, but is used by the dynamic linker (a.k.a loader) ld.so to find the shared libraries that a program or shared library requests to be loaded at runtime.

    The directories that will be searched for libraries by ld to resolve an -lfoo linkage option are, in this order of priority:

    For normal linkages done with gcc/g++ you can discover all these libraries by compiling and linking a specimen program passing the --verbose option to both the frontend and the linker, and grepping the stdout+stderr for -L, LIBRARY_PATH and SEARCH_DIR(. For example:

    $ cat file.c
int main(void)
{
    return 0;
}

$ gcc file.c -L $(pwd) --verbose -Wl,--verbose 2>&1 | egrep \(-L\|LIBRARY_PATH\|SEARCH_DIR\)
...

    I won't actually paste the output here because without grep colour-highlighting of the matches it's unrewarding. You can do it yourself. You'll notice that all the directories listed in the gcc-populated LIBRARY_PATH get added as -L <dir> options to the linkage commandline. The directories enclosed by SEARCH_DIR(<dir>) are the linker's build-configured search directories.

    Continued to address comments...

    You do not need to change the /etc/ld.so.conf.d/ to load libnf_cln.so

    The ldconfig cache and its configuration files in /etc/ld.so.conf.d/ exist to expedite the dynamic linker's search for system libraries - shared libraries that serve any application and are owned by none.

    If libnf_cln.so is a private library of your project you do not need to change the contents of /etc/ld.so.conf.d/ to let the dynamic linker find your library.

    As a rule you should resist polluting the ldconfig cache with your application's private links because the fatter it gets the slower it serves the whole system.

    Here is a worked illustration of how to avoid doing that.

    $ tail -n +1 *.c *.h
==> app.c <==
#include <private_shared_lib.h>

int main(void)
{
    private_shared_lib();
    return 0;
}

==> private_shared_lib.c <==
#include <stdio.h>

void private_shared_lib(void)
{
    puts(__FUNCTION__);
} 


==> private_shared_lib.h <==
#pragma once
void private_shared_lib(void);

$ mkdir -p app/libs

    Here is my private shared library:

    $ gcc -shared -o ./app/libs/libprivate.so private_shared_lib.c

    Link my app against it:

    $ gcc -o ./app/app app.c -I. -L./app/libs -lprivate

    Because I told the static linker -L./app/libs -lprivate, it resolved -lprivate to ./apps/libs/libprivate.so. It found that shared library; the linkage was successful and the static linker wrote the following information in the dynamic section of the output program:

    $ readelf --dynamic --wide ./app/app

Dynamic section at offset 0x2db8 contains 28 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libprivate.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 ...[irrelevant]...

    This is information for the dynamic linker to read and act on at runtime. libprivate.so is NEEDED because the static linker found that it defines the program's reference to symbol private_shared_lib. The C runtime library libc.so.6 is automatically added to the linkage by gcc and the static linker found it was NEEDED because it resolves the program's reference to symbol puts.

    Good. But:

    $ ./app/app
./app/app: error while loading shared libraries: libprivate.so: cannot open shared object file: No such file or directory

    My program fails to run because the dynamic linker looks for libprivate.so by its default search and cannot find it.

    So I will link the program again like this:

    $ gcc -o ./app/app app.c -I. -L./app/libs -lprivate -Wl,-rpath=$(pwd)/app/libs

    -Wl,<linker-option>[,linker-option,...] tells gcc to pass <linker-option>[,linker-option,...] straight through to ld. Now the dynamic section of the program contains this information:

    $ readelf --dynamic --wide ./app/app

Dynamic section at offset 0x2da8 contains 29 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libprivate.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/imk/develop/so/scrap/app/libs]
 ...[irrelevant]...

    The new information:

     0x000000000000001d (RUNPATH)            Library runpath: [/home/imk/develop/so/scrap/app/libs]

    advises the dynamic linker at runtime that /home/imk/develop/so/scrap/app/libs is an additional directory in which to search for NEEDED shared libraries, and when it knows that:

    $ ./app/app 
private_shared_lib

    my program can run.

    That's better. But I would like to install my program, with its private shared library, in an arbitrary directory (likely on another computer). Let's fake that:

    $ mkdir installdir
$ mv ./app ./installdir

    But in the installed location:

    $ ./installdir/app/app 
./installdir/app/app: error while loading shared libraries: libprivate.so: cannot open shared object file: No such file or directory

    Of course,

    0x000000000000001d (RUNPATH)            Library runpath: [/home/imk/develop/so/scrap/app/libs]

    no longer enables the dynamic linker to find libprivate.so

    So let's retreive our build directory:

    $ cp -r ./installdir/app ./

    and relink app like:

    $ gcc -o ./app/app app.c -I. -L./app/libs -lprivate -Wl,-rpath='$ORIGIN/libs'

    Now its dynamic section looks like:

    $ readelf --dynamic --wide ./app/app

Dynamic section at offset 0x2da8 contains 29 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libprivate.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/libs]
 ...[irrelevant]...

    where the dynamic linker will interpret $ORIGIN as the directory containing this file.

    Now the program runs where we built it:

    $ ./app/app
private_shared_lib

    And also if we install it elsewhere:

    $ rm -fr installdir/*
$ mv ./app ./installdir
$ ./installdir/app/app
private_shared_lib

    Lastly, if you would forsee making successive versioned releases of libnf_cln.so - libnf_cln.so.X[.Y[.Z]] - that you want to be automatically selected when your application is linked using -lnf_cln.so, then investigate linking libnf_cln.so with the option [-soname=libnf_cln.so.X] in conjunction with symbolic links:

    libnf_cln.so.so.X -> libnf_cln.so.so.X.Y [ -> libnf_cln.so.so.X.Y.Z ]

    This is the ld manual and here are its commandline options