[Boost-users] [MPI] all_gather() missing functionality or bug ?

Subject: [Boost-users] [MPI] all_gather() missing functionality or bug ?
From: Thomas Hisch (t.hisch_at_[hidden])
Date: 2012-07-25 10:47:34

Hello list,

I discovered that the all_gather function from boost.mp throws an
exception, if the local sizes of the to be gathered vector are
different:

 terminate called after throwing an instance of 'terminate called
after throwing an instance of
'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::mpi::exception>
>'
  what():  MPI_Allgather: MPI_ERR_IN_STATUS: error code in status
boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::mpi::exception>
>[thisch:14211] *** Process received signal ***

I used the following form of the all_gather function:
 all_gather(comm, myvec.data(), myvec.size(), totalvec)
where myvec is a std::vector<double>. As already mentioned the size of
myvec can be different on each processor. The documentation [0] says
that all_gather supports most uses of the MPI_Allgatherv. Therefore I
am not sure if my use of all_gather is unsupported or a just a bug.

all_gather works as excpected if myvec.size() is the same on all
processors. ATM I use the following workaround:

 { //temporary all_gather fix: use MPI_Allgatherv
    vector<int> recvcts(numprocs);
    vector<int> displs(numprocs);
    const int nrealcoeffs = N;
    const int initialcoeffsperrank = nrealcoeffs/numprocs;
    const int remainder = nrealcoeffs%numprocs;
    size_t displtmp = 0;
    for(size_t i=0; i < numprocs; i++){
      recvcts[i] = initialcoeffsperrank;
      if(i < remainder) recvcts[i]++;
      displs[i] = displtmp;
      displtmp += recvcts[i];
    }

    allgradient.resize(N);
    MPI_Allgatherv( myvec.data(), myvec.size(), MPI_DOUBLE,
                    totalvec.data(), recvcts.data(), displs.data(),
                    MPI_DOUBLE, comm);
  }

Regards
Thomas

[0] http://www.boost.org/doc/libs/1_50_0/doc/html/mpi/tutorial.html#mpi.c_mapping