Re: [boost] [serialization] fast array serialization (10x speedup)

From: David Abrahams (dave_at_[hidden])
Date: 2005-11-23 07:54:01

"Peter Dimov" <pdimov_at_[hidden]> writes:

> Matthias Troyer wrote:
>
>> Indeed this sounds like a lot of work and that's why this mechanism
>> for message passing was rarely used in the past. The hard part is to
>> manually build up the custom MPI datatype, i.e. to inform MPI about
>> what the offsets and types of the various data members in a struct
>> are.
>>
>> This is where the serialization library fits in and makes the task
>> extraordinarily easy. Saving a data member with such an MPI archive
>> will register its address, type (as well as the number of identical
>> consecutive elements in an array) with the MPI library. Thus the
>> serialization library does all the hard work already!
>
> I still don't see the 10x speedup in the subject. For a X[], the two 
> approaches are:
>
> 1. for each x in X[], "serialize" into an MPI descriptor

Do you mean "datatype?" ------------------------^^^^^^^^^^

> 2. serialize X[0] into an MPI descriptor, construct an array descriptor from 
> it

> Conceptual issues with (2) aside (the external format of X is determined by 
> X itself and you have no idea whether the structure of X[0] also describes 
> X[1]), 

I don't know what you mean.

> I'm not sure that there will be such a major speedup compared to the 
> naive (1).

Why the doubt?  We've measured.  Furthermore, as we stated earlier, in
our application we can't afford to make an in-memory copy because it
would take more than the available memory.

> Robert's point also deserves attention; a portable binary archive
> that writes directly into a socket eliminates the MPI middleman and
> will probably achieve a similar performance as your two-pass MPI
> approach.

I'll let the experts say more, but AFAICT MPI provides essential
services that aren't provided by plain sockets.  See
http://www.open-mpi.org/papers/euro-pvmmpi-2004-overview/euro-pvmmpi-2004-overview.pdf

> It also supports versioned non-PODs and other nontrivial types. 

What does?  Such an archive?

> As an example, I have a type which is saved as
>
> save( x ):
>
>     save( x.anim.name() ); // std::string
>
> and loaded as
>
> load( x ):
>
>     string tmp;
>     load( tmp );
>     x.set_animation( tmp );
>
> Not everything is field-based. 

Yes of course.  Our approach supports those types too; but you can
only take advantage of a subset of the optimizations for that part of
your communication.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com