the title may be a bit too short to be clear enough.
We have a complex C/C++-project which is built and linked in a lot of separate targets as static libraries. So my problem is that the target_link_libraries does transitive linking but in a wrong order. I need another order since we use PIMPL and the Implementations do not have any headers to include. they just include the headers of the lib which defines the PIMPL-Headers and declared the Impl-object. But for a correct linking they need to be linked after the lib which provides the header for the Pimpl. This does not happen.
This one scenario only occurs if we are building with a gcc-compiler and linker which we use for our unit-test-framework. For our target-project we use a TI clang-toolchain which provides the linker-option --reread_libs which automatically solves this problem. This is not available in gcc as far as I know.
So now to Cmake in a bit more detail. The project is too big to make a minimal example but I will try to describe the problem and shrink it to the problem:
So we have multiple static-libs inside one module:
# static libs inside the platform
add_library(memutils memutils/ForwardDeclaredStorage.hpp)
target_include_directories(memutils INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/memutils)
set_target_properties(memutils PROPERTIES LINKER_LANGUAGE CXX)
# contains the Pimpl-Headers to use, here task and mutex as extract
add_library(osal osal/mutex.hpp osal/task.hpp osal/TaskManager.hpp osal/TaskManager.cpp)
target_include_directories(osal PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/osal)
target_link_libraries(osal PUBLIC memutils)
add_library(system sys/system.hpp sys/system.cpp)
target_link_libraries(system PUBLIC osal)
target_include_directories(system PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/sys)
add_library(platformlib INTERFACE)
# example with INTERFACE keyword. Adding a dummy source and using PUBLIC does not change anything
target_link_libraries(platformlib INTERFACE osal system memutils)
# some extensions
add_library(additionslib add/additions.hpp add/additions.cpp )
target_link_libraries(additionslib PUBLIC platformlib)
target_include_directories(additionslib PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/add)
# the Impl-Lib which needs the headers
add_library(gcc_impl STATIC impls/MutexImpl.cpp impls/TaskImpl.cpp)
# so the implementations get the headers
target_link_libraries(gcc_impl PUBLIC platformlib)
# the test-case. its a project!
project(tests)
add_executable(tests test.hpp test.cpp main.cpp )
# linking all the other libs, also the test-framework (gtest), PRIVATE, PUBLIC... doesnt matter. In this case the extensions will link the platformlib transitive.
target_link_libraries(tests PUBLIC additionslib gcc_impl)
So how does the link order look? What CMake produces as a resulting link-order here is:
first it is like a "top-level-linking" and after this follow all transitive libs follow:
libadditionslib.a
libgcc_impl.a
// now the transitive libs in order of top-level-linked libs
libsystem.a
libosal.a
libmemutils.a
Now we have Problem. The platform needs the gcc_impl but it linked before the platform. I need it to be linked after the platform. If I change the generated linker-command and add the impl at last link it works fine. If not I get unreferenced symbols to the impls since they are missing then.
I tried a lot. I fiddled around with link-keywords INTERFACE PUBLIC PRIVATE in every lib. Tried to include just headers of the platform via generators instead of linking it directly. I also tried to directly link the osal which has the headers with the gcc-impl:
target_link_libraries(tests PRIVATE additionslib osal gcc_impl)
Guess what happens?
libadditionslib.a
libosal.a
libgcc_impl.a
// now the transitive libs in order of top-level-linked libs
libsystem.a
libosal.a
libmemutils.a
The effect is that the error is still the same since the system lib also needs the platform which contains the osal which needs the impl linked after it.
But no matter what I cannot change this "level"-Order of linking and force the gcc_impl to occur last.
What I need is a "in-place" linking and not a level-base linking:
libadditionslib.a
libsystem.a
libosal.a
libmemutils.a
libgcc_impl.a // gccImpl after all the platform-links!
How can I achieve this last one?
I added a minimal example that produces this linkage-error (undefined reference): https://godbolt.org/z/d19Wefrhx
you can also see that link-order:
/opt/compiler-explorer/gcc-11.3.0/bin/g++ -fdiagnostics-color=always -std=c++14 -O2 -g -L/app -Wl,-rpath,/app -Wl,-rpath,/opt/compiler-explorer/gcc-11.3.0/lib64 -Wl,-rpath,/opt/compiler-explorer/gcc-11.3.0/lib32 CMakeFiles/tests.dir/test.cpp.o CMakeFiles/tests.dir/main.cpp.o -o tests libadditionslib.a libgcc_impl.a libsystem.a libosal.a libmemutils.a
unfortunately I can't directly execute command-lines in godbolt. But I am sure if libgcc_impl.a occurs again as last one it would work. At least that's exactly how it works for us here. How can I force CMake to link the impl-library at the end?
So generally what you need to do is to link twice with both the interface and the implementaiton. Either you have to have cyclic dependency between libraries, in which case LINK_INTERFACE_MULTIPLICITY kicks in, or link twice.
You can have cyclic dependency by linking system or osal or memutils with gcc_impl. Typically what I would see is to another layer of abstraction:
target_link_libraries(osal PUBLIC memutils osal_impl)
# ...
set(CHOSEN_IMPLEMENTATION gcc) # some logic
add_library(osal_impl INTERFACE)
target_link_libraries(osal_impl INTERFACE ${CHOSEN_IMPLEMENTATION}_impl)
Then the implementation is abstract and cyclic dependency causes linking twice.
The other solution is to be explicit when linking the final stage and repeat one or the other twice:
target_link_libraries(tests PUBLIC additionslib
system gcc_impl system)
The other solution that you might be interested in on an embedded platform, is optimizations and using OBJECT libraries. That means that you have to list all dependencies on the linking stage, like so:
add_library(osal OBJECT osal/mutex.hpp osal/task.hpp osal/TaskManager.hpp osal/TaskManager.cpp)
target_include_directories(osal PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/osal)
target_link_libraries(osal PUBLIC memutils)
add_library(system OBJECT sys/system.hpp sys/system.cpp)
target_include_directories(system PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/sys)
target_link_libraries(system PUBLIC osal)
set(platformlib osal system memutils)
add_library(additionslib OBJECT add/additions.hpp add/additions.cpp )
target_include_directories(additionslib PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/add)
target_link_libraries(additionslib PUBLIC ${platformlib})
add_library(gcc_impl OBJECT impls/MutexImpl.cpp impls/TaskImpl.cpp)
target_link_libraries(gcc_impl PUBLIC ${platformlib})
add_executable(tests test.hpp test.cpp main.cpp )
target_link_libraries(tests PUBLIC ${platformlib} additionslib gcc_impl)