cmakelinkerdependencieslibraries

Linking Errors using target_link_libraries in CMake

So I was trying to go through C++ programs on a numerical platform I am evaluating but I am also getting persistent linking issues when I use CMake with this platform. The CMakeLists.txt file provided by the project developers is broken, so I opted to search out the symbols and build the example manually, ultimately the following compiled and linked correctly for me:

g++ -c dsf_main.cpp -I/usr/lib/x86_64-linux-gnu/openmpi/include \
    -I/usr/local/ga-5.8/include\
    -I/usr/local/GridPACK/include

g++ dsf_main.o \
    -L/usr/local/GridPACK/lib \
        -lgridpack_timer \
        -lgridpack_environment \
    -lgridpack_powerflow_module \
    -lgridpack_dynamic_simulation_full_y_module \
    -lgridpack_parallel \
    -lgridpack_partition \
    -lgridpack_pfmatrix_components \
    -lgridpack_ymatrix_components \
    -lgridpack_components \
    -lgridpack_stream \
    -lgridpack_block_parsers \
    -lgridpack_math \
    -lgridpack_configuration \
    -L/usr/local/ga-5.8/lib -lga++ -lga -larmci \
    -L/usr/local/boost-1.78.0/lib -lboost_mpi -lboost_serialization \
        -lboost_random \
    /usr/local/petsc-3.16.4/lib/libparmetis.so \
    /usr/local/petsc-3.16.4/lib/libpetsc.so.3.16 \
    -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi_cxx -lmpi \
    -o dsf_main.x
type here

Because I wanted to ultimately move everything to cmake I thought it should be easy to translate the above to cmake, as follows:

cmake_minimum_required(VERSION 3.21)
project(DynamicSimulation)

enable_language(CXX)

add_executable(dsf.x
   dsf_main.cpp
   )

 target_include_directories(dsf.x
   PRIVATE
   /usr/lib/x86_64-linux-gnu/openmpi/include
   /usr/local/ga-5.8/include
   /usr/local/GridPACK/include
   )

 # List of library names and their corresponding paths
 set(LIBRARIES_NAMES
  "mpi"
  "mpi_cxx"
  "petsc"
  "parmetis"
  "boost_random"
  "boost_serialization"
  "boost_mpi"
  "armci"
  "ga"
  "ga++"
  "gridpack_configuration"
  "gridpack_math"
  "gridpack_block_parsers"
  "gridpack_stream"
  "gridpack_components"
  "gridpack_ymatrix_components"
  "gridpack_pfmatrix_components"
  "gridpack_partition"
  "gridpack_parallel"
  "gridpack_dynamic_simulation_full_y_module"
  "gridpack_powerflow_module"
  "gridpack_environment"
  "gridpack_timer"
)

set(LIBRARIES_FOUND)

foreach(LIBRARY_NAME IN LISTS LIBRARIES_NAMES)
  find_library(LIBRARY_PATH NAMES ${LIBRARY_NAME}
    PATHS /usr/local/GridPACK/lib
    /usr/local/ga-5.8/lib
    /usr/local/boost-1.78.0/lib
    /usr/local/petsc-3.16.4/lib
    /usr/lib/x86_64-linux-gnu/openmpi/lib
    NO_CACHE
    )

    # Check if the library was found
    if(NOT LIBRARY_PATH)
        message(FATAL_ERROR "Library '${LIBRARY_PATH}' could not be found.")
    endif()

    # Append the found library path to the LIBRARIES_FOUND list
    list(APPEND LIBRARIES_FOUND ${LIBRARY_PATH})
    unset(LIBRARY_PATH)
endforeach()

message(STATUS "All Library PATHS: ${LIBRARIES_FOUND}")

target_link_libraries(dsf.x PUBLIC ${LIBRARIES_FOUND})

I reversed the library order compared to direct linking because this is how target_link_libraries is documented to work, what comes later depends on what comes first. However this again gave me linking errors with unresolved symbols and I don't understand why, at all. If the direct linking and compilation works then this should work because its a replica.

More surprising , if I repeat the libraries 3 times ! (it has to be 3, 2 times doesn't work ) , like so:

target_link_libraries(dsf.x PUBLIC ${LIBRARIES_FOUND} ${LIBRARIES_FOUND} ${LIBRARIES_FOUND})

Things work perfectly , the program compiles and links correctly. This doesn't make ANY sense to me. There is something I feel I don't understand fundamentally about Cmake. Can someone tell what what is happening here and point me in the right direction ?

Thanks !

Solution

  • I reversed the library order compared to direct linking because this is how target_link_libraries is documented to work, what comes later depends on what comes first. However this again gave me linking errors with unresolved symbols and I don't understand why, at all.

    Here you are under a misconception. You reversed the linkage order of the libraries for CMake because you believe that "what comes later depends on what comes first", which is the opposite of the order required for the GNU linker ld and the g++ frontend (which invokes ld on your behalf to perform linkages).

    This is not so. You believe that the documentation of target_link_libraries specifies that its <item> libraries must be listed with items depended-on before those that depend on them, but it doesn't - you must have misconstrued or misremembered the documentation. If you regenerate your failing CMake build (without the repetitions of ${LIBRARIES_FOUND}) and run make VERBOSE=1 you will observe that the build output following Linking CXX executable dsf.x shows your libraries being input to the g++ linkage in the order of your set(LIBRARIES_NAMES ...). That's would you'd expect. But this is the reverse of the successful order of your manual g++ linkage, and a failed linkage is what you'd expect of g++ when you do that. Dependees must come before the libraries they depend on for g++, and likewise for CMake when g++ is calling the linkage.

    If the direct linking and compilation works then this should work because its a replica.

    It doesn't work because it's not a replica. If you put the libraries in set(LIBRARIES_NAMES...) back in the successful order then the CMake linkage will also succeed, without needing any repetitions of ${LIBRARIES_FOUND} in your target_link_libraries directive.

    More surprising , if I repeat the libraries 3 times ! (it has to be 3, 2 times doesn't work )... [t]hings work perfectly... This doesn't make ANY sense to me.

    Here's how it makes sense. I'll make gross but serviceable simplifications of the GNU linker's modus operandi in this illustration, noting that it is a single-pass linker by default. Say you're linking a program prog and the sole object file in the linkage is file.o. It depends on a library liba, which depends on libb, which depends on libc. It doesn't matter whether these are shared libraries (*.so) or static archives (*.a). There are no other dependency relationships.

    Case 1.

    If the linker inputs are specified as:

    file.o -lc -lb -la

    with the libraries in reverse order of dependency, then:

    There are no more input files. The dependency of liba on libb is unsatisfied and the linkage fails with undefined references in liba.

    Case 2.

    If the linker inputs are specified as:

    file.o -lc -lb -la -lc -lb -la

    with the libraries in reverse order of dependency, twice, then:

    There are no more input files. The dependency of libb on libc is unsatisfied and the linkage fails with undefined references in libb.

    Case 3.

    If the linker inputs are specified as:

    file.o -lc -lb -la -lc -lb -la -lc -lb -la

    with the libraries in reverse order of dependency, thrice, then:

    There are no more input files. All input files have at last been linked, in the order: file.o (input #1), liba (input #4), libb (input #6), libc (input #8). The linkage is successful.

    It takes 3 repetitions of the library list to pull off the linkage in this case as in yours, but the number 3 is not magic. If libraries are input in the wrong order, the necessary number of repetitions of the libraries will be influenced by the number of the libraries, the particular symbolic dependencies between them and between them and the preceding object files and the disordered sequence in which they are input. But there will necessarily be some finite number of repetitions that resolves all dependencies if the linkage can succeed at all. At least one of the libraries will be linked at 1st sight because some object file already linked must depend on at least one library. If a 2nd repetition is necessary, then at least a 2nd library must be linked at 2nd sight because at least one library linked at 1st sight must depend on at least one library not linked at 1st sight. And so on, until sufficiently many repetitions pull all the necessary libraries into the linkage.

    Cases may exist in which two or more libraries on which a program depends are interdependent (liba -> libb -> liba) or cyclically interdependent (liba -> libb -> libc -> liba), so that no non-repeating ordering can satisfy all the dependencies in single-pass linkage. For the GNU linker, linkage with such interdependent libraries can be achieved by deliberately repeating the list of interdependent libraries as many times as required. More simply, for static libraries, it can be achieved by enclosing that list within --start-group/--end-group linker options, which causes the linker (expensively) to iterate symbol resolution passes over the list until no new undefined references arise. For shared libraries, it can be achieved by the linker option --no-as-needed, which causes the linker to link subsequent shared libraries unconditionally.