I'm just learning to use coarry with Fortran. I have a very simple program. I have an array of length 9 that I want to distribute over 3 processes, do some calculations, and merge them back to a single array (basic MPI_scatter/MPI_gather like problem).
program name
implicit none
integer:: arr(9), local(3)[*]
integer:: i, j, k, iz
arr = [(i, i = 1, 9)]
local(:)[1] = arr(1:3)
local(:)[2] = arr(4:6)
local(:)[3] = arr(7:9)
iz = this_image()
local = local*iz
sync all
if(iz == 1) then
arr(1:3) = local(:)[1]
arr(4:6) = local(:)[2]
arr(7:9) = local(:)[3]
write(*,'(*(i3))')arr
endif
end program name
I'm compiling this with
gfortran -fcorray=lib -lcaf_mpi
and executing with this
mpirun -np 3 ./a.out
This should a print output like this
1 2 3 8 10 12 21 24 27
But, If I run the executable multiple times (without recompilation) sometimes it shows multiple results, like 1 2 3 4 5 6 21 24 27 etc. that is the values do not get updated with the calculation.
What I'm doing wrong here? How to fix this problem
Welcome to the wonderful world of shared memory programming!
You have a race condition. It is possible that one of the images is so far behind the others that it resets the (remote) data after the other image performs local = local*iz. Here's a way to fix it, being extra careful that only 1 process ever sets up a given part of the data:
ijb@ijb-Latitude-5410:~/work/stack$ cat caf.f90
program name
implicit none
integer:: arr(9), local(3)[*]
integer:: i, j, k, iz
iz = this_image()
! Only one image should ever write to a given memory location
! between synchronisation points. In a real code each different image
! would set up different parts of the array
If( iz == 1 ) Then
arr = [(i, i = 1, 9)]
local(:)[1] = arr(1:3)
local(:)[2] = arr(4:6)
local(:)[3] = arr(7:9)
End If
! Make sure the array is fully set up before you use it
sync all
local = local*iz
sync all
if(iz == 1) then
arr(1:3) = local(:)[1]
arr(4:6) = local(:)[2]
arr(7:9) = local(:)[3]
write(*,'(*(i3))')arr
endif
end program name
ijb@ijb-Latitude-5410:~/work/stack$ mpif90 -std=f2018 -fcheck=all -Wall -Wextra -O -g -fcoarray=lib caf.f90 -lcaf_openmpi
caf.f90:4:17:
4 | integer:: i, j, k, iz
| 1
Warning: Unused variable ‘j’ declared at (1) [-Wunused-variable]
caf.f90:4:20:
4 | integer:: i, j, k, iz
| 1
Warning: Unused variable ‘k’ declared at (1) [-Wunused-variable]
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
1 2 3 8 10 12 21 24 27