gcccmakempiopensusedrake

Integration of drake to OpenSUSE - Algorithm/LinearSolvers/IpMumpsSolverInterface.cpp:28:10: fatal error: mpi.h: No such file or directory


I am trying to integrate drake to OpenSUSE. This is the custom GitHub repo containing my progress.

Despite installed openmpi-devel and exporting the location of my openmpi package location:

dan@74ff521a7dbe:~/drake/cmake>  find /usr/ -wholename *openmpi 
/usr/lib64/mpi/gcc/openmpi
/usr/lib64/mpi/gcc/openmpi/lib64/openmpi
/usr/lib64/mpi/gcc/openmpi/share/openmpi
/usr/lib64/mpi/gcc/openmpi/include/openmpi
/usr/share/doc/packages/openmpi
/usr/share/modules/gnu-openmpi
dan@74ff521a7dbe:~/drake/cmake> echo $MPI_HOME
/usr/lib64/mpi/gcc/openmpi
dan@74ff521a7dbe:~/drake/cmake> echo $PATH
/home/dan/.pyenv/shims:/home/dan/.pyenv/bin:/usr/lib64/mpi/gcc/openmpi/bin:/home/dan/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
dan@74ff521a7dbe:~/drake/cmake> echo $LD_LIBRARY_PATH
/usr/lib64/mpi/gcc/openmpi/lib:/usr/lib/llvm-14/lib:

and working exemplary code:

dan@74ff521a7dbe:~> cat hello_mpi.c
#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
    // Initialize the MPI environment
    MPI_Init(NULL, NULL);

    // Get the number of processes
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    // Get the rank of the process
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

    // Get the name of the processor
    char processor_name[MPI_MAX_PROCESSOR_NAME];
    int name_len;
    MPI_Get_processor_name(processor_name, &name_len);

    // Print off a hello world message
    printf("Hello world from processor %s, rank %d out of %d processors\n",
           processor_name, world_rank, world_size);

    // Finalize the MPI environment.
    MPI_Finalize();
}
dan@74ff521a7dbe:~> mpicc -o hello_mpi hello_mpi.c
dan@74ff521a7dbe:~> mpiexec -np 4 ./hello_mpi
Unexpected end of /proc/mounts line `overlay / overlay rw,relatime,lowerdir=/var/lib/docker/overlay2/l/H2ZGGNXKPSBKC5LUJAFIZT2I3U:/var/lib/docker/overlay2/l/KO3UVFH5AQX23H7UQ3CTT7MU72:/var/lib/docker/overlay2/l/SJ2UWCEGI72H7WS5B5DRLKZZ24:/var/lib/docker/overlay2/l/UBUZXCO4DMYISSSTRKZXKDTY2F:/var/lib/docker/overlay2/l/RZFBQFEWB2KT5SFVVRS35BWZWH:/var/lib/docker/overlay2/l/4NEUA3URWZ5J7E4PUQC6UJYQFR:/var/lib/docker/overlay2/l/CC23IOZFBZRXHSKZTP2RGL2XXA:/var/lib/docker/overlay2/l/BKOQ5AX3DBF3HRL7EYRIIXSJH2:/var/lib/docker/overlay2/l/HEQ44C36MNTNE'
Unexpected end of /proc/mounts line `NKOCL63BWEHUU:/var/lib/docker/overlay2/l/ZUQNI5STPEOBL6CYNCAPDH4NCP:/var/lib/docker/overlay2/l/UBIAXBHVXDTZ7UCVAG5BIG6F33:/var/lib/docker/overlay2/l/7XY6WOR36GVPP4BVKFTWAP6DKK:/var/lib/docker/overlay2/l/A6NW2WQQJR5KEFC2LP2RBAMPI6:/var/lib/docker/overlay2/l/W2FRL65ZMNI6IW2OAU64G6YUKO:/var/lib/docker/overlay2/l/IF47FTPHFWAKVH4U2BMGUMHOTS:/var/lib/docker/overlay2/l/FNRJUOUX6FNMG52AAMBVQQSDYI,upperdir=/var/lib/docker/overlay2/8b2b2266c58c9ac96813642be3abb3c0d4cac6191f3d4076fdf2cd2397c429d6/diff,workdir=/var/li'
--------------------------------------------------------------------------
[[17328,1],0]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
  Host: 74ff521a7dbe

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
Hello world from processor 74ff521a7dbe, rank 1 out of 4 processors
Hello world from processor 74ff521a7dbe, rank 2 out of 4 processors
Hello world from processor 74ff521a7dbe, rank 3 out of 4 processors
Hello world from processor 74ff521a7dbe, rank 0 out of 4 processors
[74ff521a7dbe:34210] 3 more processes have sent help message help-mpi-btl-base.txt / btl:no-nics
[74ff521a7dbe:34210] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

I get the following error:

dan@74ff521a7dbe:~/drake/cmake> cmake .. -DWITH_USER_FMT:BOOLEAN=ON -DWITH_USER_SPDLOG:BOOLEAN=ON -DCMAKE_PREFIX_PATH=/usr/local
CMake Warning at CMakeLists.txt:60 (message):
  Could NOT find the lsb_release executable


fatal: not a git repository: /home/dan/drake/../.git/modules/drake
INFO: Reading 'startup' options from /home/dan/drake/cmake/drake_build_cwd/.bazelrc: --output_base=/home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Reading rc options for 'info' from /home/dan/drake/tools/bazel.rc:
  Inherited 'common' options: --enable_bzlmod=false
INFO: Reading rc options for 'info' from /home/dan/drake/cmake/drake_build_cwd/.bazelrc:
  Inherited 'common' options: --announce_rc=no --repo_env=CC=/usr/bin/gcc-11 --repo_env=CXX=/usr/bin/g++-11
INFO: Reading rc options for 'info' from /home/dan/drake/tools/bazel.rc:
  Inherited 'build' options: -c opt --strip=never --strict_system_includes --cxxopt=-std=c++20 --host_cxxopt=-std=c++20 --test_output=errors --test_summary=terse --test_tag_filters=-gurobi,-mosek,-snopt --test_env=DISPLAY --test_env=GRB_LICENSE_FILE --test_env=MOSEKLM_LICENSE_FILE --test_env=LCM_DEFAULT_URL=memq:// --test_env=MPLBACKEND=Template
INFO: Reading rc options for 'info' from /home/dan/drake/tools/ubuntu.bazelrc:
  Inherited 'build' options: --force_pic --fission=dbg --features=per_object_debug_info --action_env=PATH=/usr/bin:/bin --action_env=PYTHONNOUSERSITE=1 --test_env=PYTHONNOUSERSITE=1
INFO: Reading rc options for 'info' from /home/dan/drake/cmake/drake_build_cwd/.bazelrc:
  Inherited 'build' options: --symlink_prefix=/ --color=yes --subcommands=no --package_path=%workspace%:/home/dan/drake --notrim_test_configuration --action_env=CCACHE_DISABLE=1 --config=Release
INFO: Found applicable config definition build:Release in file /home/dan/drake/cmake/drake_build_cwd/.bazelrc: --compilation_mode=opt
bazel-bin: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3/execroot/drake/bazel-out/k8-opt/bin
bazel-genfiles: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3/execroot/drake/bazel-out/k8-opt/bin
bazel-testlogs: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3/execroot/drake/bazel-out/k8-opt/testlogs
character-encoding: file.encoding = ISO-8859-1, defaultCharset = ISO-8859-1
command_log: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3/command.log
committed-heap-size: 566MB
execution_root: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3/execroot/drake
gc-count: 41
gc-time: 2806ms
install_base: /home/dan/.cache/bazel/_bazel_dan/install/117cee491f5c7d83be6e3c6d6b5b8ca4
java-home: /home/dan/.cache/bazel/_bazel_dan/install/117cee491f5c7d83be6e3c6d6b5b8ca4/embedded_tools/jdk
java-runtime: OpenJDK Runtime Environment (build 11.0.6+10-LTS) by Azul Systems, Inc.
java-vm: OpenJDK 64-Bit Server VM (build 11.0.6+10-LTS, mixed mode) by Azul Systems, Inc.
max-heap-size: 8304MB
output_base: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3
output_path: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3/execroot/drake/bazel-out
package_path: %workspace%:/home/dan/drake
release: release 6.4.0
repository_cache: /home/dan/.cache/bazel/_bazel_dan/cache/repos/v1
server_log: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3/java.log.74ff521a7dbe.dan.log.java.20240325-134945.413
server_pid: 413
used-heap-size: 340MB
workspace: /home/dan/drake/cmake/drake_build_cwd
-- Configuring done
-- Generating done
-- Build files have been written to: /home/dan/drake/cmake
dan@74ff521a7dbe:~/drake/cmake> make install
INFO: Analyzed target //:install (138 packages loaded, 15486 targets configured).
INFO: Found 1 target...
ERROR: /home/dan/.cache/bazel/_bazel_/7ea85213ca47cc67b02970ecea7788b3/external/ipopt_internal_fromsource/BUILD.bazel:347:16: Compiling external/ipopt_internal_fromsource/drake_src/Algorithm/LinearSolvers/IpMumpsSolverInterface.cpp failed: (Exit 1): gcc-11 failed: error executing command (from target @ipopt_internal_fromsource//:_build) /usr/bin/gcc-11 -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 79 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
bazel-out/k8-opt/bin/external/ipopt_internal_fromsource/drake_src/Algorithm/LinearSolvers/IpMumpsSolverInterface.cpp:28:10: fatal error: mpi.h: No such file or directory
   28 | #include "mpi.h"
      |          ^~~~~~~
compilation terminated.
Target //:install failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2410.761s, Critical Path: 342.39s
INFO: 2296 processes: 153 internal, 2143 processwrapper-sandbox.
FAILED: Build did NOT complete successfully
make[2]: *** [CMakeFiles/drake_cxx_python.dir/build.make:70: CMakeFiles/drake_cxx_python] Error 1
make[1]: *** [CMakeFiles/Makefile2:839: CMakeFiles/drake_cxx_python.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

This is the dockerfile that I use:

# Use OpenSUSE Leap 15.4 as base
FROM opensuse/leap:15.4

# Install vim, sudo, and other necessary tools and libraries for drake
RUN zypper --non-interactive update && \
    zypper --non-interactive install -y vim sudo git cmake make java-1_8_0-openjdk-devel \
    glib2-devel lapack-devel libX11-devel ocl-icd-devel opencl-headers patch patchelf \
    pkg-config python3-devel python3-pygame zlib-devel pkg-config eigen3-devel \
    libmumps5 mumps-devel gcc-fortran nasm wget unzip tar gzip openmpi-devel

# Install GCC 11
RUN zypper --non-interactive install -y gcc11 gcc11-c++

# Set GCC 11 as the default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 110 --slave /usr/bin/g++ g++ /usr/bin/g++-11 && \
    update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 35 && \
    update-alternatives --set cc /usr/bin/gcc && \
    update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 35 && \
    update-alternatives --set c++ /usr/bin/g++

# Download and build fmt with position-independent code
RUN wget https://github.com/fmtlib/fmt/archive/refs/tags/8.1.1.tar.gz && \
    tar -xzf 8.1.1.tar.gz && \
    cd fmt-8.1.1 && \
    mkdir build && cd build && \
    cmake .. -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DCMAKE_INSTALL_PREFIX=/usr/local && \
    make && \
    sudo make install

# Download and build spdlog with position-independent code, using external fmt
RUN wget https://github.com/gabime/spdlog/archive/refs/tags/v1.10.0.tar.gz && \
    tar -xzf v1.10.0.tar.gz && \
    cd spdlog-1.10.0 && \
    mkdir build && cd build && \
    cmake .. -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DSPDLOG_FMT_EXTERNAL=ON -DCMAKE_INSTALL_PREFIX=/usr/local && \
    make && \
    sudo make install

# Install Bazelisk
RUN curl -L https://github.com/bazelbuild/bazelisk/releases/download/v1.7.5/bazelisk-linux-amd64 > /usr/local/bin/bazelisk && \
    chmod +x /usr/local/bin/bazelisk && \
    ln -s /usr/local/bin/bazelisk /usr/local/bin/bazel

# Download and place jchart2d-3.3.2.jar in /usr/share/java/jchart2d.jar
RUN wget https://repo1.maven.org/maven2/net/sf/jchart2d/jchart2d/3.3.2/jchart2d-3.3.2.jar -O /usr/share/java/jchart2d.jar

# Create .pc files for LAPACK and BLAS
RUN echo -e "prefix=/usr\nexec_prefix=\${prefix}\nlibdir=\${exec_prefix}/lib64\nincludedir=\${prefix}/include\n\nName: LAPACK\nDescription: Linear Algebra Package\nVersion: 3.9.0\nLibs: -L\${libdir} -llapack\nCflags: -I\${includedir}" > /usr/share/pkgconfig/lapack.pc && \
    echo -e "prefix=/usr\nexec_prefix=\${prefix}\nlibdir=\${exec_prefix}/lib64\nincludedir=\${prefix}/include\n\nName: BLAS\nDescription: Basic Linear Algebra Subprograms\nVersion: 3.9.0\nLibs: -L\${libdir} -lblas\nCflags: -I\${includedir}" > /usr/share/pkgconfig/blas.pc

# Set up the environment for user 'dan'
RUN groupadd -r dan && \
    useradd -m -s /bin/bash -r -g dan dan && \
    mkdir -p /home/dan/drake && \
    chown -R dan:dan /home/dan/drake && \
    echo "dan ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

USER dan
ENV HOME /home/dan
WORKDIR $HOME

# Install dependencies for pyenv and Python build
RUN sudo zypper --non-interactive install -y git gcc make zlib-devel libbz2-devel libopenssl-devel readline-devel \
    sqlite3-devel python3-clang15 clang15-devel

# Install pyenv and Python 3.10
RUN git clone https://github.com/pyenv/pyenv.git $HOME/.pyenv && \
    echo 'export PYENV_ROOT="$HOME/.pyenv"' >> $HOME/.bashrc && \
    echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> $HOME/.bashrc && \
    echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n  eval "$(pyenv init --path)"\nfi' >> $HOME/.bashrc && \
    /bin/bash -c "source $HOME/.bashrc && pyenv install 3.10.0 && pyenv global 3.10.0"

# Ensure the selected Python version 3.10 is used and install PyYAML
RUN /bin/bash -c "source $HOME/.bashrc && pyenv rehash && pip install PyYAML"

# Set environment variables for GCC to ensure Bazel uses the correct version
ENV CC=/usr/bin/gcc-11
ENV CXX=/usr/bin/g++-11
ENV PYENV_ROOT $HOME/.pyenv
ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH
ENV LD_LIBRARY_PATH=/usr/lib/llvm-14/lib:$LD_LIBRARY_PATH
ENV CMAKE_PREFIX_PATH=/usr/local/lib/cmake:${CMAKE_PREFIX_PATH}

# Additional steps which ensure that we have everything needed for openmpi
RUN sudo zypper --non-interactive install -y openmpi-devel

ENV MPI_HOME=/usr/lib64/mpi/gcc/openmpi
ENV PATH="${MPI_HOME}/bin:${PATH}"
ENV LD_LIBRARY_PATH="${MPI_HOME}/lib:${LD_LIBRARY_PATH}"

Additional potentially helpful information:

https://gist.github.com/hedaniel7/1df5b3931047bddd4b75a0d6d63f2077 In this gist I have listed all official openmpi packages that have a relation to mumps. If they might be helpful, I would need to know further details about the version constraints from drake about them.


Solution

  • The file that's needed is not part of any real MPI runtime. Mumps ships its own MPI stub file, which is what Drake needs.

    On Ubuntu:

    $ dlocate mpi.h | grep mumps
    libmumps-headers-dev: /usr/include/mumps_seq/mpi.h
    
    $ cat /usr/include/mumps_seq/mpi.h
    /*
     *
     *  This file is part of MUMPS 5.4.1, released
    ...
    /* This is the minimum to have the C interface to MUMPS work with the
     * C example provided. Other stub functions of the MPI standard may be
     * added if needed. */
    
    typedef LIBSEQ_INT MPI_Comm; /* Simple type for MPI communicator */
    static MPI_Comm MPI_COMM_WORLD=(MPI_Comm)0;
    ...
    

    Your error is probably that the MUMPS install is somehow either misconfigured, or broken, or needs some other kind of help.

    My suggestion is to take a look at the Debian (or Ubuntu) packages' contents for MUMPS, and make sure you have something on par with that in SUSE, one way or another.