bazelopensusedrakefmtspdlog

Integration of drake to OpenSUSE - build error with spdlog and fmt

I have created this GitHub repository with a OpenSUSE 15.4 docker file in it, where I am trying to integrate the Drake simulation for OpenSUSE. I tried to run in the Docker image the following build steps in the Drake repository:

I get the following very long error traceback involving spdlog and fmt:

https://gist.github.com/hedaniel7/9b74a2b2c6a6ec625e94ab85cc0f395d

Here is a small sample:

ERROR: /home/dan/drake/common/BUILD.bazel:307:17: Compiling common/find_resource.cc failed: (Exit 1): gcc-10 failed: error executing command (from target //common:_find_resource_compiled_cc_impl) /usr/bin/gcc-10 -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 71 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
In file included from external/fmt/include/_usr_local_include/spdlog/fmt/fmt.h:28,
                 from external/fmt/include/_usr_local_include/spdlog/common.h:50,
                 from external/fmt/include/_usr_local_include/spdlog/spdlog.h:12,
                 from bazel-out/k8-opt/bin/common/_virtual_includes/essential/drake/common/text_logging.h:85,
                 from common/find_resource.cc:17:
external/fmt/include/_usr_local_include/spdlog/fmt/bundled/core.h:308:30: error: redefinition of 'struct fmt::v10::type_identity<T>'
  308 | template <typename T> struct type_identity { using type = T; };
      |                              ^~~~~~~~~~~~~
In file included from external/fmt/include/_usr_local_include/fmt/format.h:54,
                 from common/find_resource.cc:10:
external/fmt/include/_usr_local_include/fmt/base.h:301:30: note: previous definition of 'struct fmt::v10::type_identity<T>'
  301 | template <typename T> struct type_identity {
      |                              ^~~~~~~~~~~~~
In file included from external/fmt/include/_usr_local_include/spdlog/fmt/fmt.h:28,
                 from external/fmt/include/_usr_local_include/spdlog/common.h:50,
                 from external/fmt/include/_usr_local_include/spdlog/spdlog.h:12,
                 from bazel-out/k8-opt/bin/common/_virtual_includes/essential/drake/common/text_logging.h:85,
                 from common/find_resource.cc:17:
external/fmt/include/_usr_local_include/spdlog/fmt/bundled/core.h:325:8: error: redefinition of 'struct fmt::v10::monostate'

The problem may be related to the warning

DEBUG: /home/dan/drake/tools/workspace/pkg_config.bzl:240:18: pkg-config of fmt returned an include path that contains /usr/local/include that may contain unrelated headers
DEBUG: /home/dan/drake/tools/workspace/pkg_config.bzl:240:18: pkg-config of fmt returned an include path that contains /usr/local/include that may contain unrelated headers
DEBUG: /home/dan/drake/tools/workspace/pkg_config.bzl:240:18: pkg-config of spdlog returned an include path that contains /usr/local/include that may contain unrelated headers

which I get in the above terminal interaction gist

This is the Dockerfile I used (also viewable in the GitHub repository)

# Use OpenSUSE Leap 15.4 as base
FROM opensuse/leap:15.4

# Install vim, sudo, and other necessary tools and libraries for drake
RUN zypper --non-interactive update && \
    zypper --non-interactive install -y vim sudo git cmake make java-1_8_0-openjdk-devel \
    glib2-devel lapack-devel libX11-devel ocl-icd-devel opencl-headers patch patchelf \
    pkg-config python3-devel python3-pygame zlib-devel pkg-config eigen3-devel \
    libmumps5 mumps-devel gcc-fortran nasm wget llvm-clang llvm-clang-devel

# Add the repository for GCC 10
RUN zypper addrepo -f http://download.opensuse.org/repositories/devel:/gcc/openSUSE_Leap_15.4/ devel_gcc && \
    zypper --non-interactive --gpg-auto-import-keys refresh && \
    zypper --non-interactive install -y gcc10 gcc10-c++

# Set GCC 10 as the default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 --slave /usr/bin/g++ g++ /usr/bin/g++-10 && \
    update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30 && \
    update-alternatives --set cc /usr/bin/gcc && \
    update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30 && \
    update-alternatives --set c++ /usr/bin/g++

# Clone, build, and install fmt and spdlog from source
RUN cd /tmp && \
    git clone https://github.com/fmtlib/fmt.git && \
    cd fmt && mkdir build && cd build && cmake .. && make install && \
    cd /tmp && \
    git clone https://github.com/gabime/spdlog.git && \
    cd spdlog && mkdir build && cd build && cmake .. && make install

# Install Bazelisk
RUN curl -L https://github.com/bazelbuild/bazelisk/releases/download/v1.7.5/bazelisk-linux-amd64 > /usr/local/bin/bazelisk && \
    chmod +x /usr/local/bin/bazelisk && \
    ln -s /usr/local/bin/bazelisk /usr/local/bin/bazel

# Download and place jchart2d-3.3.2.jar in /usr/share/java/jchart2d.jar
RUN wget https://repo1.maven.org/maven2/net/sf/jchart2d/jchart2d/3.3.2/jchart2d-3.3.2.jar -O /usr/share/java/jchart2d.jar

# Create .pc files for LAPACK and BLAS
RUN echo -e "prefix=/usr\nexec_prefix=\${prefix}\nlibdir=\${exec_prefix}/lib64\nincludedir=\${prefix}/include\n\nName: LAPACK\nDescription: Linear Algebra Package\nVersion: 3.9.0\nLibs: -L\${libdir} -llapack\nCflags: -I\${includedir}" > /usr/share/pkgconfig/lapack.pc && \
    echo -e "prefix=/usr\nexec_prefix=\${prefix}\nlibdir=\${exec_prefix}/lib64\nincludedir=\${prefix}/include\n\nName: BLAS\nDescription: Basic Linear Algebra Subprograms\nVersion: 3.9.0\nLibs: -L\${libdir} -lblas\nCflags: -I\${includedir}" > /usr/share/pkgconfig/blas.pc

# Set up the environment for user 'dan'
RUN groupadd -r dan && \
    useradd -m -s /bin/bash -r -g dan dan && \
    mkdir -p /home/dan/drake && \
    chown -R dan:dan /home/dan/drake && \
    echo "dan ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

# Switch to user 'dan' before installing pyenv and Python
USER dan
ENV HOME /home/dan
WORKDIR $HOME

# Install dependencies for pyenv and Python build
RUN sudo zypper --non-interactive install -y git gcc make zlib-devel libbz2-devel libopenssl-devel readline-devel sqlite3-devel tar gzip


# Install pyenv and Python 3.10
RUN git clone https://github.com/pyenv/pyenv.git $HOME/.pyenv && \
    echo 'export PYENV_ROOT="$HOME/.pyenv"' >> $HOME/.bashrc && \
    echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> $HOME/.bashrc && \
    echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n  eval "$(pyenv init --path)"\nfi' >> $HOME/.bashrc && \
    /bin/bash -c "source $HOME/.bashrc && pyenv install 3.10.0 && pyenv global 3.10.0"

# Ensure the selected Python version 3.10 is used and install PyYAML
RUN /bin/bash -c "source $HOME/.bashrc && pyenv rehash && pip install PyYAML"

# Set environment variables for GCC to ensure Bazel uses the correct version
ENV CC=/usr/bin/gcc-10
ENV CXX=/usr/bin/g++-10
ENV PYENV_ROOT $HOME/.pyenv
ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH

I built spdlog and fmt from source from GitHub repositories to circumvent an issue before where Bazel complained that it didn't find it.

I did a little test, where I wrote a minimal C++-program with spdlog and fmt to see if it compiles and runs and it does:

dan@fae637d1a972:~> vim spdlogfmtHelloworld.cpp

#include <spdlog/spdlog.h>
#include <fmt/core.h>

int main() {
    // Using fmt to format a string
    std::string name = "World";
    std::string formatted = fmt::format("Hello, {}!", name);

    // Using spdlog to log the formatted string
    spdlog::info(formatted);

    return 0;
}
dan@fae637d1a972:> g++ main.cpp -o myprogram -lspdlog -lfmt
g++: error: main.cpp: No such file or directory
dan@fae637d1a972:> g++ spdlogfmtHelloworld.cpp -o myprogram -lspdlog -lfmt
dan@fae637d1a972:> ./myprogram
[2024-03-07 13:17:25.889] [info] Hello, World!

Side note with OpenSUSE libfmt8 package

Steps to reproduce the error:

Solution

  • When building Drake via CMake with user-provided fmt and spdlog, the typical approach is to instruct Drake's CMake code to use find_package() to locate those dependencies, i.e.,

    cmake \
    -DWITH_USER_FMT:BOOLEAN=ON \
    -DWITH_USER_SPDLOG:BOOLEAN=ON

    This skips the pkg-config logic that would otherwise happen by default inside of Drake.

    In fact, it would be most typical to have an overall CMakeLists project build those instead of using the Dockerfile (as shown in https://github.com/RobotLocomotion/drake-external-examples/blob/main/drake_cmake_external/CMakeLists.txt ) but that's not strictly required.

    The main point is to turn on "WITH_USER_FMT" etc., and be sure that you've build fmt and spdlog in a way that CMake will find (or you've set CMAKE_PREFIX_PATH to help find them).

    Looking at the gist output, the other problem is that your spdlog build is using a bundled copy of fmt. You need to configure your spdlog build to use the copy of fmt you already built, as in -DSPDLOG_FMT_EXTERNAL:BOOLEAN=ON (see here).

    Big picture: if SUSE already has any build of spdlog or fmt, it would be ideal to use those instead of building from source. Possibly WITH_USER_FMT etc with an appropriate CMake prefix path would find the SUSE ones.