Re: [boost] [serialization] fast array serialization (10x speedup)

From: Matthias Troyer (troyer_at_[hidden])
Date: 2005-11-23 03:13:40

On Nov 23, 2005, at 6:00 AM, Robert Ramey wrote:

> Ian McCulloch wrote:
>
>> You need to construct an MPI 'datatype', which is conceptually a
>> record of how the type is laid out as a sequence of (offset,
>> nested_datatype) pairs.
>
> ...
>
>> The 'usual' way (if there is such a thing; datatypes seem to be not
>> used
>> much in MPI) of constructing datatypes is using offsetof(), or just
>> knowing what the layout is for compiler X, and constructing the
>> datatype by hand.
>
> ...
>> I don't know what mechanism Dave is thinking of to construct the
>> datatype,
>> it sounds unlikely that it could be done via a usual serialization
>> function (but maybe if you could do member pointer arithmetic to
>> replace
>> offsetof() ?).
>
> Sounds like a lot of work. But if one want's to use all this stuff  
> - then
> what's the point of using the serialization library at all? I mean
> I can see using one or the other - but what is the point of both?

Indeed this sounds like a lot of work and that's why this mechanism  
for message passing was rarely used in the past. The hard part is to  
manually build up the custom MPI datatype, i.e. to inform MPI about  
what the offsets and types of the various data members in a struct are.

This is where the serialization library fits in and makes the task  
extraordinarily easy. Saving a data member with such an MPI archive  
will register its address, type (as well as the number of identical  
consecutive elements in an array) with the MPI library. Thus the  
serialization library does all the hard work already!

As Dave mentioned earlier, this information can then used by the MPI  
library (and network hardware) to directly serialize the data into  
the I/O buffers of the network interconnect, without ever creating a  
copy in memory, and automatically taking care of potential endianness  
and format issues on heterogeneous networks.

To answer your question: one wants to use both since the  
serialization library is used to create the information which the MPI  
library needs to efficiently send the data.

Matthias