[Boost-mpi] Support for MPI_IN_PLACE option in reduce operation

Subject: [Boost-mpi] Support for MPI_IN_PLACE option in reduce operation
From: Notargiacomo Thibault (gnthibault_at_[hidden])
Date: 2014-05-04 06:44:18

Dear Boost::mpi developers

I am new to MPI, but not to boost in general, and as I have seen that
boost::mpi has builtin support for boost::serialization, I decided to
directly implement my code using boost::mpi.

Doing so, I spotted what I think to be a mismatch between MPI
specifications and boost::mpi API, in the reduce operation.

 -----What I have:
I have a huge buffer of float/double containing differents values from
one rank to another :

float array[arraysize]; // each mpi process contains an instance of it

 -----What I want to do:
I want each element of the buffer of rank0 to contain the sum
(element-wise) of each instance of this buffer in other process, ie,
perform a reduction with std::plus<float>() operator:

// Perform reduction in master
mpi::reduce( my_Boost_MPI_Communicator, array, arraysize, result,
std::plus<volT>(), 0 );
// Perform reduction in slave
mpi::reduce( my_Boost_MPI_Communicator, array, arraysize,
std::plus<volT>(), 0 );

In order to have:

result[0] = Sum(array_0[0], array_1[0], ..., array_(N-1)[0]);
result[1] = Sum(array_0[1], array_1[1], ..., array_(N-1)[1]);
...
result[arraysize-1] = Sum(array_0[arraysize-1], ..., array_(N-1)[arraysize-1]);

 -----What is not working:
As the array buffer is huge, I don't want to allocate another result
buffer of the same size, this is why I tried to use the same pointer
in both in and out argument.

-----The error I got:

terminate called after throwing an instance of
'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::mpi::exception>
>'
  what():  MPI_Reduce: MPI_ERR_ARG: invalid argument of some other kind

----- What MPI says about it

If I was using Raw MPI, I should use MPI_Reduce with MPI_IN_PLACE as
the buffer argument in the master, and the buffer pointer in the
result argument.

------ Mismatch ( ? ):
The only thig I have seen about it in boost documentation seems to
result from this ticket:
https://svn.boost.org/trac/boost/ticket/6704
And only targets the all_reduce function.

In addition, I tried the all_reduce function, in the buffer fashion,
while omitting the result buffer pointer, and the compiler was not
even capable of finding a corresponding definition:
mpi::all_reduce( my_Boost_MPI_Communicator, array, arraysize,
std::plus<volT>() );

Could you please give me some indication about what is wrong with
either what I want to do, or the way I want to use boost::mpi ?

Thank you very much in advance.

kind regards

Thibault