I am starting from a C++ codebase which depends upon Eigen, and I am attempting to see if it can be combined with the Kokkos library to add GPU functionality, so I need both of these to be found and linked against in the CMake file.
Additionally my Eigen dependency was already using BLAS and LAPACK.
If I include either of these dependencies in the target_link_libraries section, then all is well, however if I include both at once then the compilation fails with
In file included from /main.cpp:6:
In file included from /bask/apps/live/EL8-ice/software/Eigen/3.4.0-GCCcore-11.3.0/include/Eigen/Dense:7:
In file included from /bask/apps/live/EL8-ice/software/Eigen/3.4.0-GCCcore-11.3.0/include/Eigen/Eigenvalues:55:
/bask/apps/live/EL8-ice/software/Eigen/3.4.0-GCCcore-11.3.0/include/Eigen/src/Eigenvalues/SelfAdjointEigenSolver_LAPACKE.h:80:1: error: __host__ function 'compute' cannot overload __host__ __device__ function 'compute'
80 | EIGEN_LAPACKE_EIG_SELFADJ(double, double, double, dsyev)
| ^
/bask/apps/live/EL8-ice/software/Eigen/3.4.0-GCCcore-11.3.0/include/Eigen/src/Eigenvalues/SelfAdjointEigenSolver_LAPACKE.h:77:9: note: expanded from macro 'EIGEN_LAPACKE_EIG_SELFADJ'
77 | EIGEN_LAPACKE_EIG_SELFADJ_2(EIGTYPE, LAPACKE_TYPE, LAPACKE_RTYPE, LAPACKE_NAME, ColMajor ) \
| ^
/bask/apps/live/EL8-ice/software/Eigen/3.4.0-GCCcore-11.3.0/include/Eigen/src/Eigenvalues/SelfAdjointEigenSolver_LAPACKE.h:43:72: note: expanded from macro 'EIGEN_LAPACKE_EIG_SELFADJ_2'
43 | SelfAdjointEigenSolver<Matrix<EIGTYPE, Dynamic, Dynamic, EIGCOLROW> >::compute(const EigenBase<InputType>& matrix, int options) \
| ^
/bask/apps/live/EL8-ice/software/Eigen/3.4.0-GCCcore-11.3.0/include/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:216:29: note: previous declaration is here
216 | SelfAdjointEigenSolver& compute(const EigenBase<InputType>& matrix, int options = ComputeEigenvectors);
| ^
This error seems to imply that Eigen is attempting to use device (GPU?) functionality.
I have a minimal reproduction of the code here, a single main.cpp file and the CMake script:
main.cpp:
#define lapack_complex_float std::complex<float>
#define lapack_complex_double std::complex<double>
#define EIGEN_USE_BLAS
#define EIGEN_USE_LAPACKE
#include <Eigen/Dense>
int main(void){
Eigen::Matrix3d::Zero()
}
CMakeList.txt:
cmake_minimum_required (VERSION 3.8)
set(CMAKE_CXX_STANDARD 20)
project ("trial")
find_package(Kokkos REQUIRED)
find_package(BLAS REQUIRED)
find_package(LAPACK REQUIRED)
find_package(Eigen3 REQUIRED NO_MODULE)
add_executable(main "main.cpp")
target_link_libraries(
main
Kokkos::kokkos
Eigen3::Eigen
BLAS::BLAS
LAPACK::LAPACK
)
Note that this assumes Kokkos, BLAS and Eigen are already installed and findable.
It will work if I remove the link to Kokkos, however adding the Kokkos::kokkos link (even without invoking any Kokkos code) causes the compilation to fail.
FYI I was using:
This error seems to imply that Eigen is attempting to use device (GPU?) functionality.
Eigen has some CUDA support in terms of marking their functions as __host__ __device__ when detecting a CUDA compiler so one can use them in device code, i.e. they do not provide actual kernels.
As Kokkos with CUDA backend will compile .cpp files with a CUDA compiler, I played around a bit with the given code on cuda.godbolt.org using NVCC 11.8 which gives different warnings and errors (The errors can be avoided by passing --expt-relaxed-constexpr to NVCC.).
Finally I tried using Clang as the CUDA compiler and I got the same errors reported in the question. I.e. Kokkos (or rather its CMake package) defaults to using Clang as the CUDA compiler instead of NVCC if one doesn't set CMAKE_CXX_COMPILER explicitly.
I don't think Eigen's CUDA support is tested with Clang as the CUDA compiler and Clang handles __host__ and __device__ slightly differently to NVCC which might be the reason for this particular problem, i.e. this issue is not directly connected to Kokkos at all and it seems to be known at Eigen.
As OP does not seem to intend using Eigen in GPU code, I would recommend just turning off the Eigen CUDA support by defining EIGEN_NO_CUDA.
I'm not sure if it is possible to combine Eigen's CUDA support with Kokkos' CUDA backend*, but if that is the aim, one has to get the Kokkos CMake package to use NVCC (or rather Kokkos' nvcc_wrapper) over Clang as its CUDA compiler:
-DCMAKE_CXX_COMPILER=<Kokkos Install Directory>/bin/nvcc_wrapper
as described in the Kokkos documentation.
*: It doesn't really make sense to do so as one would lose some of the portability given by Kokkos, but I see no reason for it not to work with nvcc_wrapper and maybe even for HIP with some hacking.