linuxgcclinkerldlinker-flags

gcc 11 on Ubuntu 22.02 not linking library constructors

How can I figure out why gcc is failing to link my library (with only a lib constructor) when built with GCC 11 on Ubuntu 22.04?

I set up a tiny example here to try to replicate the problem: https://godbolt.org/z/PT4jETToj , the issue is that I can't - on godbolt it works. The problem only occurs locally.

The library constructor is suppose to print Library constructor called!, notice how it does when built in a container, but not when built on my host (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0)

❯ rm -rf main.gcc; docker run -it --rm -v $(pwd):/workspace gcc:11.4 bash -c "cd /workspace; ./build.sh" && ./main.gcc
Building GCC variant
Library constructor called!
Main program started!

❯ rm -rf main.gcc; ./build.sh && ./main.gcc
Building GCC variant
Main program started!

where build.sh runs:

g++ -fPIC -shared -o libexample-gcc.so example.cpp \
  && g++ -std=c++17 -o main.gcc main.cpp -L. -lexample-gcc -Wl,-rpath,.

This also works if I build it locally in clang. I was starting to think my system was simply broken, but my colleague's system (20.04, gcc 9.4) also doesn't run the library constructor.

The issue appears to be between linking and runtime. When I add the -Wl,--verbose flag I can see that the linker sees libexample for both gcc and clang

# verbose linker log
attempt to open ./libexample-gcc.so succeeded
./libexample-gcc.so

But the resulting gcc binary on my host doesn't show libexample in the ldd output, whereas it does when it's built with clang. And since it's not in the linker list, it's not present at runtime to run.

The symbol is well defined in the library

❯ nm -C libexample-gcc.so | rg library_constructor
0000000000001179 T library_constructor()

but the library is completely missing in the ldd:

❯ ( LD_LIBRARY_PATH=$(pwd) ldd main.gcc | rg example ) || echo "not found"
not found

❯ LD_LIBRARY_PATH=$(pwd) ldd main.clang | rg example
        libexample-clang.so => /home/matt/workspace/lib_constructor_example/libexample-clang.so (0x00007d0188775000)

This is true even if I explicitly add --no-as-needed, i.e.

g++ -std=c++17 -o main.gcc main.cpp -L. -lexample-gcc -Wl,-rpath,. -Wl,--no-as-needed

Could there be something in my environment or in my system setup that is controlling this?

Solution

  • Your shared library is not needed by the linker because, at the point when the linker considers it in the input sequence, it has accumulated 0 unresolved references to the one symbol library_constructor() that your library defines (since main.o does not reference it, of course). Your library resolves nothing when input, so it is not needed.

    A library constructor would normally be a static function, so as not to be referencable from other modules. I'm sure you plan to add to the library some external definitions of functions or data objects to make it useful. If you do that, and any of these symbols is referenced in main.o, or in any other object file (including ones extracted from static libraries) that is input to the linkage before your library, then your library will be needed and it will be linked whatever frontend you use, on whatever system.

    Until it provides any symbols that the linkage needs, whether your library is linked or not depends on whether the boilerplate linkage options which are generated behind the scenes by the particular frontend that you invoke include -Wl,-as-needed before the commandline libraries are interpolated.

    When --as-needed is in effect, the linker will consider a shared library to be needed if and only if it really is needed, to resolve references accrued earlier in the linkage. When --as-needed is not in effect the linker will consider the library needed, and link it, even it isn't actually needed to resolve references.

    In the scenarios where library_constructor() is called and ldd reports your library dynamically linked, the frontend in use is configured not to default --as-needed. In the scenarios where library_constructor() is not called and ldd does not report your library, the frontend in use does default --as-needed

    Your effort to coerce --no-as-needed with:

    g++ -std=c++17 -o main.gcc main.cpp -L. -lexample-gcc -Wl,-rpath,. -Wl,--no-as-needed

    is unsuccessful because each of the contrary linker options --as-needed/--no-as-needed is operative upon shared libraries subsequently input until and unless the contrary option appears. So your -Wl,--no-as-needed comes too late, since -lexample-gcc has already been input. It will be operative only on any shared libraries input in g++'s boilerplate additions to the commandline, unless that boilerplate appends --as-needed afterwards, but before any standard shared libaries are appended (which in fact it doesn't manage).

    These options are targetted to a particular shared library or limited sequence of shared libraries in this way:

    -Wl,--as-needed -lfoo ... -Wl,--no-as-needed

    or, more cautiously:

    -Wl,--push-state,--as-needed -lfoo ... -Wl,--pop-state

    Your success in linking your library locally with clang++ is due to the fact that your local clang(++), unlike your local gcc\g++, does not default --as-needed. The same is true for me:

    $ g++ --version
g++ (Ubuntu 13.2.0-23ubuntu4) 13.2.0
...
$ clang++ --version
Ubuntu clang version 18.1.6 (++20240518023429+1118c2e05e67-1~exp1~20240518143527.144)

$ tail -n +1 main.cpp example.cpp CMakeLists.txt 
==> main.cpp <==
#include <iostream>

auto main() -> int {
    std::cout << "Main program started!" << std::endl;
    return 0;
}

==> example.cpp <==
#include <iostream>

// Constructor function
static __attribute__((constructor)) void library_constructor() {
    std::cout << "Library constructor called!" << std::endl;
}

==> CMakeLists.txt <==
cmake_minimum_required(VERSION 3.21)

project(main-test VERSION 1.0 LANGUAGES CXX)
set(CMAKE_VERBOSE_MAKEFILE TRUE)
set(CXX_STANDARD 17)

add_library(example SHARED)
target_sources(example PRIVATE example.cpp)

add_executable(main)
target_sources(main PRIVATE main.cpp)
target_link_libraries(main PRIVATE example)

$ mkdir build
$ cd build/
$ cmake ..
...
$ make VERBOSE=1 | grep '\-o main'
/usr/bin/c++ CMakeFiles/main.dir/main.cpp.o -o main  -Wl,-rpath,/home/imk/develop/so/scrap2/build libexample.so

    Last thing there is the g++ linkage commandline. Let's redo it --verbose and see if --as-needed is in the boilerplate:

    $ /usr/bin/c++ CMakeFiles/main.dir/main.cpp.o -o main  --verbose -Wl,-rpath,/home/imk/develop/so/scrap2/build libexample.so 2>&1 | grep -o '\--as-needed'; echo Done
--as-needed
Done

    Yes. And:

    $ readelf --dynamic --wide main | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

    libexample.so was not NEEDED:

    $ ./main
Main program started!

    Redo with clang++:

    $ /usr/bin/clang++ CMakeFiles/main.dir/main.cpp.o -o main  --verbose -Wl,-rpath,/home/imk/develop/so/scrap2/build libexample.so 2>&1 | grep -o '\--as-needed'; echo Done
Done

    No --as-needed. And:

    $ readelf --dynamic --wide main | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libexample.so]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

    Three shared unncessary shared libraries linked, including libexample.so.

    $ ./main
Library constructor called!
Main program started!