I'm trying to distribute a 2D matrix over a number of processors with some overlapping regions. I noticed that the function scatterv of mpi4py doesn't work with 2d arrays, I was forced then to use 1D arrays. The problem here is that the displacements in some processors can be greater than the limit of int. How can I solve this problem please?
import numpy as np
from mpi4py import MPI
from math import ceil
# Dimensions of the matrix
N = 50000
M = 50000
comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()
n = ceil(N/size)
offset = N-(size-1)*n # Offset
start = np.zeros(size, dtype=np.float64) # Start rows of the chunks
end = np.zeros(size, dtype=np.float64) # End rows of the chunks
sz_loc = np.zeros(size, dtype=np.float64) # Size of he local chunk
for i in range(size):
pp = n
if i == size - 1:
pp = offset
start[i] = max(0, i * n - mf)
end[i] = min(N, i * n + pp + mf)
sz_loc[i] = (end[i]-start[i])*M
if rank == 0:
Im = np.array(np.round(10*np.random.rand(N, M)),
dtype=np.float32).ravel()
else:
Im = None
Im_loc = np.zeros((int(sz_loc[rank]),), dtype=np.float32)
comm.Barrier()
comm.Scatterv([Im, sz_loc, start*M, MPI.FLOAT], Im_loc, root=0)
Here is the error I get, using for the example 10 processors
comm.Scatterv([Im, sz_loc, start*M, MPI.FLOAT], Im_loc, root=0)
File "mpi4py/MPI/Comm.pyx", line 626, in mpi4py.MPI.Comm.Scatterv
File "mpi4py/MPI/msgbuffer.pxi", line 538, in mpi4py.MPI._p_msg_cco.for_scatter
File "mpi4py/MPI/msgbuffer.pxi", line 440, in mpi4py.MPI._p_msg_cco.for_cco_send
File "mpi4py/MPI/msgbuffer.pxi", line 313, in mpi4py.MPI.message_vector
File "mpi4py/MPI/asarray.pxi", line 22, in mpi4py.MPI.chkarray
File "mpi4py/MPI/asarray.pxi", line 15, in mpi4py.MPI.getarray
OverflowError: value too large to convert to int
Finally I definded a new data type using Create_contiguous() function.