c++boostmpiopenmpiboost-mpi

Open MPI Broadcast Latency Measurement


From reading the documentation, MPI_Bcast is a blocking call (and therefore boost::mpi::broadcast) is as well. Is measuring the amount of time it takes the root node to broadcast a good measure of the time it takes for the data to get from the root node to all the other nodes?

I.e.

int64_t t1 = Utility::picosecondTime(); //exactly what it sounds like; utility that measures current time in picoseconds
boost::mpi::broadcast(communicator, variable, 0);
std::cout << "Broadcast took " << Utility::picosecondTime()-t1 << std::endl;

Or for straight OpenMPI:

MPI_Comm comm; 
int array[100]; 
...
int64_t t1 = Utility::picosecondTime(); 
MPI_Bcast( array, 100, MPI_INT, 0, comm); 
std::cout << "Broadcast took " << Utility::picosecondTime()-t1 << std::endl;

Solution

  • An MPI_BCAST is usually implemented in some sort of tree fashion where processes at the top of the tree can exit the algorithm after they've finished their part of the broadcast. So if rank 0 sends messages to ranks 1 and n/2, then it can leave after those messages are done. So the answer to your question is: no, that's not an accurate measurement.

    Without accurately synchronized clocks, it's difficult to actually measure the time that a full broadcast takes across all nodes. There are tricks you can do to get closer (use an MPI_BARRIER to synchronize before taking the start time and use the longest time spent by any process in the broadcast), but as clocks still tend to have some drift, nothing will be perfect.