clangc++20ldubuntu-22.04

linker: main_tests.cpp.o file not recognized; file format


My c++20 CMake-based project builds fine and links the build files just fine normally as well. However, I seem to get this error sporadically:

CMakeFiles/reinforce_tests_all.dir/test/main_tests.cpp.o: file not recognized: file format not recognized

I have had this problem before and managed to get rid of it via cleaning the build and rebuilding. This time around it is a persistent problem for me though. The full error log reads (with linker verbose output):

: && /usr/bin/clang++ -g -v CMakeFiles/reinforce_tests_all.dir/test/main_tests.cpp.o CMakeFiles/reinforce_tests_all.dir/test/libreinforce/test_gridworld.cpp.o CMakeFiles/reinforce_tests_all.dir/test/libreinforce/test_space_box.cpp.o CMakeFiles/reinforce_tests_all.dir/test/libreinforce/test_space_discrete.cpp.o CMakeFiles/reinforce_tests_all.dir/test/libreinforce/test_space_multidiscrete.cpp.o -o reinforce_tests_all -L/home/root/conan_cache/p/range0301bf3d76d5d/p/lib   -L/home/root/conan_cache/p/b/fmted8c40f5dd02d/p/lib   -L/home/root/conan_cache/p/b/spdlo0e2ce77e2e884/p/lib   -L/home/root/conan_cache/p/pybin895b9560d7d3f/p/lib   -L/home/root/conan_cache/p/b/gtest79f180a2a4833/p/lib -Wl,-rpath,/home/root/conan_cache/p/range0301bf3d76d5d/p/lib:/home/root/conan_cache/p/b/fmted8c40f5dd02d/p/lib:/home/root/conan_cache/p/b/spdlo0e2ce77e2e884/p/lib:/home/root/conan_cache/p/pybin895b9560d7d3f/p/lib:/home/root/conan_cache/p/b/gtest79f180a2a4833/p/lib  libreinforce++.a  /home/root/conan_cache/p/b/spdlo0e2ce77e2e884/p/lib/libspdlogd.a  /usr/lib/aarch64-linux-gnu/libpython3.10.so  /home/root/conan_cache/p/b/gtest79f180a2a4833/p/lib/libgtestd.a  -lpthread  /home/root/conan_cache/p/b/fmted8c40f5dd02d/p/lib/libfmtd.a  -lm  /usr/lib/aarch64-linux-gnu/libpython3.10.so && :
Ubuntu clang version 16.0.6 (++20231112100510+7cbf1a259152-1~exp1~20231112100554.106)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/aarch64-linux-gnu/11
Found candidate GCC installation: /usr/bin/../lib/gcc/aarch64-linux-gnu/13
Selected GCC installation: /usr/bin/../lib/gcc/aarch64-linux-gnu/13
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/bin/ld" -pie -EL -z relro --hash-style=gnu --build-id --eh-frame-hdr -m aarch64linux -dynamic-linker /lib/ld-linux-aarch64.so.1 -o reinforce_tests_all /lib/aarch64-linux-gnu/Scrt1.o /lib/aarch64-linux-gnu/crti.o /usr/bin/../lib/gcc/aarch64-linux-gnu/13/crtbeginS.o -L/home/root/conan_cache/p/range0301bf3d76d5d/p/lib -L/home/root/conan_cache/p/b/fmted8c40f5dd02d/p/lib -L/home/root/conan_cache/p/b/spdlo0e2ce77e2e884/p/lib -L/home/root/conan_cache/p/pybin895b9560d7d3f/p/lib -L/home/root/conan_cache/p/b/gtest79f180a2a4833/p/lib -L/usr/bin/../lib/gcc/aarch64-linux-gnu/13 -L/lib/aarch64-linux-gnu -L/usr/lib/aarch64-linux-gnu -L/lib -L/usr/lib CMakeFiles/reinforce_tests_all.dir/test/main_tests.cpp.o CMakeFiles/reinforce_tests_all.dir/test/libreinforce/test_gridworld.cpp.o CMakeFiles/reinforce_tests_all.dir/test/libreinforce/test_space_box.cpp.o CMakeFiles/reinforce_tests_all.dir/test/libreinforce/test_space_discrete.cpp.o CMakeFiles/reinforce_tests_all.dir/test/libreinforce/test_space_multidiscrete.cpp.o -rpath /home/root/conan_cache/p/range0301bf3d76d5d/p/lib:/home/root/conan_cache/p/b/fmted8c40f5dd02d/p/lib:/home/root/conan_cache/p/b/spdlo0e2ce77e2e884/p/lib:/home/root/conan_cache/p/pybin895b9560d7d3f/p/lib:/home/root/conan_cache/p/b/gtest79f180a2a4833/p/lib libreinforce++.a /home/root/conan_cache/p/b/spdlo0e2ce77e2e884/p/lib/libspdlogd.a /usr/lib/aarch64-linux-gnu/libpython3.10.so /home/root/conan_cache/p/b/gtest79f180a2a4833/p/lib/libgtestd.a -lpthread /home/root/conan_cache/p/b/fmted8c40f5dd02d/p/lib/libfmtd.a -lm /usr/lib/aarch64-linux-gnu/libpython3.10.so -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/bin/../lib/gcc/aarch64-linux-gnu/13/crtendS.o /lib/aarch64-linux-gnu/crtn.o
CMakeFiles/reinforce_tests_all.dir/test/main_tests.cpp.o: file not recognized: file format not recognized
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Other solutions here typically suggest to do make clean, but as I said this is not solving the problem for me. Inspecting the file also shows nothing suspicious as far as I can tell (but I am also not well versed in the ways of the linkers):

╰─ file reinforce_tests_all.dir/test/main_tests.cpp.o                                                                
reinforce_tests_all.dir/test/main_tests.cpp.o: LLVM IR bitcode

╰─ /usr/bin/ld --version                                                                                                                                   ─╯
GNU ld (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

I am working on macos (arm64) but operating inside an arm64 Ubuntu 22.04 docker image with a GCC-13 installation and a clang-16 installation (which is also my cxx compiler).

I hope you have suggestions on how to investigate/fix this problem, would be greatly appreciated!

EDIT: Here the compile_commands.json segment particularly for main_tests.cpp:

{
  "directory": "/tmp/reinforce/cmake-build-debug-docker-clang-reinforce",
  "command": "/usr/bin/clang++ -DHAVE_CBLAS=1 -DSPDLOG_ACTIVE_LEVEL=SPDLOG_LEVEL_DEBUG -DSPDLOG_COMPILED_LIB -DSPDLOG_FMT_EXTERNAL -DXTENSOR_USE_XSIMD -I/tmp/reinforce/src/libreinforce/include -I/tmp/reinforce/test/shared_test_utils -isystem /home/root/conan_cache/p/pcg-c4aa2cafcf36e3/p/include -isystem /home/root/conan_cache/p/range0301bf3d76d5d/p/include -isystem /home/root/conan_cache/p/b/fmted8c40f5dd02d/p/include -isystem /home/root/conan_cache/p/xtens8c3f32033d144/p/include -isystem /home/root/conan_cache/p/xtl7161fe009dc21/p/include -isystem /home/root/conan_cache/p/nlohm0567ffc90cfc1/p/include -isystem /home/root/conan_cache/p/xsimd7b0628df207bb/p/include -isystem /home/root/conan_cache/p/b/spdlo0e2ce77e2e884/p/include -isystem /home/root/conan_cache/p/froze7ac551bf2fb61/p/include -isystem /tmp/reinforce/cmake-build-debug-docker-clang-reinforce/_deps/xtensor-python-src/include -isystem /home/root/conan_cache/p/pybin895b9560d7d3f/p/include -isystem /usr/include/python3.10 -isystem /usr/local/lib/python3.10/dist-packages/numpy/core/include -isystem /tmp/reinforce/cmake-build-debug-docker-clang-reinforce/_deps/xtensor-blas-src/include -isystem /home/root/conan_cache/p/b/gtest79f180a2a4833/p/include -g -std=gnu++20 -fcolor-diagnostics -g3 -ftemplate-backtrace-limit=0 -flto=auto -funroll-loops -Wall -Wextra -Wshadow -Wnon-virtual-dtor -Wold-style-cast -Wcast-align -Wunused -Woverloaded-virtual -Wpedantic -Wconversion -Wsign-conversion -Wnull-dereference -Wdouble-promotion -Wformat=2 -Winvalid-pch -Xclang -include-pch -Xclang /tmp/reinforce/cmake-build-debug-docker-clang-reinforce/CMakeFiles/reinforce_tests_gridworld.dir/cmake_pch.hxx.pch -Xclang -include -Xclang /tmp/reinforce/cmake-build-debug-docker-clang-reinforce/CMakeFiles/reinforce_tests_gridworld.dir/cmake_pch.hxx -o CMakeFiles/reinforce_tests_gridworld.dir/test/main_tests.cpp.o -c /tmp/reinforce/test/main_tests.cpp",
  "file": "/tmp/reinforce/test/main_tests.cpp",
  "output": "CMakeFiles/reinforce_tests_gridworld.dir/test/main_tests.cpp.o"
},

/Michael


Solution

  • The problem arose from using Clang's Link Time Optimization feature (LTO). Apparently, LTO is more compiler specific than I expected: In this mode clang outputs LLVM IR bitcode files and not ELF object files. Since GNU ld cannot work with LLVM IR it crashes. I switched to LLVM's lld to handle the LTO part. To activate these settings as well do:

        target_compile_options(
                MyTarget
                PUBLIC  # or qualifier of your choice here 
                "-flto"
        )
    
        target_link_options(
                MyTarget
                PUBLIC  # or qualifier of your choice here 
                "-fuse-ld=lld"
        )