Using mpi4py, I have created a code which defines a new datatype to hold a matrix's column and send it to other MPI process:
column = MPI.INT.Create_vector(4, 1, 4)
column.Commit()
if rank == 0:
matrix = np.array([[1, 2, 3, 4],
[5, 6, 7, 8],
[9, 10, 11, 12],
[13, 14, 15, 16]], dtype=np.intc)
comm.Send([matrix, 1, column], 1)
else:
matrix = np.array([[-1, -1, -1, -1],
[-2, -2, -2, -2],
[-3, -3, -3, -3],
[-4, -4, -4, -4]], dtype=np.intc)
comm.Recv([matrix, 1, column], source=0)
print(matrix)
This works, as long as only the first column of the matrix is concerned. The program prints:
[[ 1 -1 -1 -1]
[ 5 -2 -2 -2]
[ 9 -3 -3 -3]
[13 -4 -4 -4]]
How do I go about sending a second column of matrix from process 0 to process 1? When I try to send it using comm.Send([matrix[0,1], 1, column], 1), the receiving process inserts values [2, -136678432, 0, 0] into its matrix. Thus I believe that somehow I access ndarray's memory in a wrong manner.
If I translate this code to C and send using MPI_Send (&matrix[0][1], 1, column, 1, 123, MPI_COMM_WORLD) everything works and second column is transferred to process 1. How can I make the code work in Python/mpi4py? Please also note that I'd prefer the solution to still use the vector MPI type as declared in the first line (column = MPI.INT.Create_vector(4, 1, 4)).
My solution was to alter the Send part in my sample code as follows: comm.Send([np.frombuffer(matrix.data, np.intc, offset=4), 1, column], 1)
After experimenting, I have found that Send has problems reading from memory buffer when it's given as matrix[0,1]. We have to explicitly tell it to read from memory held by matrix (matrix.data part) and give an offset into that memory. As numpy by default stores the data in C order, we have to move 4 bytes ahead.