Hi Lorenz,

Thank you very much! Your code works! But it seems I have to manually allocate the memory before unpacking the data from the archive? Then if my data is a vector of vector, does it mean I have to resize every vector as well?

Back to the MPI error, sorry I was wrong to say that it is related to boost.MPI. With the new code passing MPI_PACKED data, I am receiving the same out of memory error. So it seems my MPI installation has a problem in the message size for packed data. Thanks to both Lorenz and Alain!

Jiaxin

2015-11-19 15:56 GMT+00:00 Lorenz Hübschle-Schneider <huebschle@kit.edu>:

Hi,

On 19/11/15 16:07, Jiaxin Han wrote:

Hi,

Could anyone point me to an example of mixing boost serialization with C
MPI functions?

you need to send the archive size first if it's not precisely known. Try something like this:

mpi::packed_oarchive oa(comm);
oa << sendbuf;
auto sendptr = const_cast<void*>(oa.address());
// cast to int because MPI uses ints for sizes like it's still 1990
int sendsize = static_cast<int>(oa.size());
MPI_Send(&sendsize, 1, MPI_INT, 1, 0, comm);
MPI_Send(sendptr, sendsize, MPI_PACKED, 1, 0, comm);

The receiving side would look similar:

mpi::packed_iarchive ia(comm);
int recvsize;
MPI_Recv(&recvsize, 1, MPI_INT, 0, 0, comm, MPI_STATUS_IGNORE);
ia.resize(recvsize);
auto recvptr = ia.address();
MPI_Recv(recvptr, recvsize, MPI_PACKED, 0, 0, comm, MPI_STATUS_IGNORE);

Do I have to make sure buffer is big enough or the oarchive will handle
the memory automatically? If the former, what is the correct memory size
to hold a vector? I guess it should hold not just vec.data(), but also
vec.size().

It does this automatically for the sending size. On the receiver, you have to know how much data to expect and reserve enough memory accordingly. This is what the first transmission in my code above is for.

And lastly, oa does not appear to be the correct variable to pass to
MPI_Send. Then what should I pass to MPI_Send after creating the archive?

You pass in the pointer and its length -- oa.address() and oa.size() in the example above.

I am asking because the boost mpi installation on our server appears to
have a limitation on the message size. For example, the attached code
reports an error as:

terminate called after throwing an instance of
'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::mpi::exception>
 >'
   what():  MPI_Alloc_mem: MPI_Alloc_mem: Out of "special" (shared) memory
MPI Application rank 0 killed before MPI_Finalize() with signal 6

This appears to be a problem only with the boost installation on the
server. The code runs correctly on my local machine.

I'm no expert but to me this sounds like an issue with the MPI installation on your servers. Note that it's not Boost.MPI that's giving you an error - the error is from the C function that it calls.

Cheers,
Lorenz