I am currently working on migrating some Fortran code over to cudaFortran. Specifically the task involves the spectral analysis of massive matrices in order to diagonalize them. Here's the code I've fabricobbled so far
program main
!Trials for usage of cusovlerDn<t>syevd for spectral analysis of a symmetric matrix, see http://docs.nvidia.com/cuda/cusolver/index.html#syevd-example1 for the example used as a base
!Compilation example: 'pgf90 Main.cuf -lcusolver -Mcuda=cuda8.0'
use cudafor !has to go first
use cusolverDn
implicit none
integer :: info
integer,parameter :: q2 = SELECTED_REAL_KIND(15,305)
real(q2), device, dimension(3,3) :: A_d
real(q2), dimension(3,3) :: A
real(q2), device, dimension(3) :: W_d
real(q2), dimension(3) :: W
integer :: stat, lwork, m, lda
real(q2), device, allocatable :: work_d(:)
integer, device :: devInfo
type(cusolverDnHandle) :: h
stat=cusolverDnCreate(h)
W_d=(/0,0,0/)
print *, stat
m=3
lda = m
A_d(1,1:3)=(/4,1,2/)
A_d(2,1:3)=(/1,-1,1/)
A_d(3,1:3)=(/2,1,3/) !eigenvalues are 5.84947, 1.44865, -1.29812
! A_d(1,1:3)=(/1,0,0/)
! A_d(2,1:3)=(/0,1,0/)
! A_d(3,1:3)=(/0,0,1/)
stat=cusolverDnDsyevd_bufferSize(h, CUSOLVER_EIG_MODE_NOVECTOR, CUBLAS_FILL_MODE_UPPER, m, A_d, lda, W_d, lwork)
print *, stat
allocate(work_d(lwork))
stat=cusolverDnDsyevd(h, CUSOLVER_EIG_MODE_NOVECTOR, CUBLAS_FILL_MODE_UPPER, m, A_d, lda, W_d, work_d, lwork, devInfo)
print *, stat !returns 6 as if there was an error
info=devInfo
print *, info !devInfo returns 0, as if the operation was successful
stat=cudaDeviceSynchronize()
print *, stat
W=W_d
print *, W
A=A_d
print *, A
deallocate(work_d)
stat=cusolverDnDestroy(h)
print *, stat
end program main
Compilation and mem-check output are as follows:
olafur@olafur-X556UQK:~/Skyrmions2017/Project$ pgf90 Main.cuf -lcusolver -Mcuda=cuda8.0
olafur@olafur-X556UQK:~/Skyrmions2017/Project$ cuda-memcheck ./a.out
========= CUDA-MEMCHECK
0
0
========= Program hit cudaErrorInvalidDeviceFunction (error 8) due to "invalid device function" on CUDA API call to cudaLaunch.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/usr/lib/x86_64-linux-gnu/libcuda.so.1 [0x2ef503]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x5b906e]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2e0857]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2e0270]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2e3df3]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2e1720]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2e0157]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 (cusolverDnDsytrd + 0x37) [0x2e3f17]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2ea607]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2eb744]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 (cusolverDnDsyevd + 0x27) [0x2ea157]
========= Host Frame:./a.out [0x1b2d]
========= Host Frame:./a.out [0x1514]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0xf0) [0x20830]
========= Host Frame:./a.out [0x13f9]
=========
6
========= Program hit cudaErrorInvalidDeviceFunction (error 8) due to "invalid device function" on CUDA API call to cudaGetLastError.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/usr/lib/x86_64-linux-gnu/libcuda.so.1 [0x2ef503]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x5b6793]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2e1727]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2e0157]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 (cusolverDnDsytrd + 0x37) [0x2e3f17]
0
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2ea607]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 [0x2eb744]
========= Host Frame:/opt/pgi/linux86-64/2017/cuda/8.0/lib64/libcusolver.so.8.0 (cusolverDnDsyevd + 0x27) [0x2ea157]
========= Host Frame:./a.out [0x1b2d]
0
========= Host Frame:./a.out [0x1514]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0xf0) [0x20830]
========= Host Frame:./a.out [0x13f9]
=========
0.000000000000000 0.000000000000000 0.000000000000000
4.000000000000000 1.000000000000000 2.000000000000000
1.000000000000000 -1.000000000000000 1.000000000000000
2.000000000000000 1.000000000000000 3.000000000000000
0
========= ERROR SUMMARY: 2 errors
It looks like I'm not actually invoking the cusolverDnDsyevd functions properly, most likely I am not using the right types of variables. But since I am semi-illiterate in programming and the only example I have to follow is written in C (using those fancy void** things) I don't know what is proper.
EDIT: Full output of deviceQuery
olafur@olafur-X556UQK:~/NVIDIA_CUDA-8.0_Samples/1_Utilities/deviceQuery$ ./deviceQuery
./deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)
Device 0: "GeForce 940MX"
CUDA Driver Version / Runtime Version 8.0 / 8.0
CUDA Capability Major/Minor version number: 5.0
Total amount of global memory: 2002 MBytes (2099642368 bytes)
( 3) Multiprocessors, (128) CUDA Cores/MP: 384 CUDA Cores
GPU Max Clock rate: 1242 MHz (1.24 GHz)
Memory Clock rate: 900 Mhz
Memory Bus Width: 64-bit
L2 Cache Size: 1048576 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 1 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = GeForce 940MX
Result = PASS
Since the code works fine on another system at my disposal the problem was indeed a runtime environment issue, as suggested by Robert Crovella
Moral of the story: Always try at least 2 systems.