Re: [boost] [serialization] fast array serialization (10x speedup)

From: Ian McCulloch (ianmcc_at_[hidden])
Date: 2005-11-22 20:34:29

Peter Dimov wrote:

> David Abrahams wrote:
> 
>>> Peter Dimov wrote:
> 
>>>> In the situations that I've needed it, the subset of types that
>>>> could be written in a single operation could not be described by a
>>>> type trait. A type trait can't tell you whether the in-memory
>>>> representation and the on-disk representation of a type are the
>>>> same.
>>
>> As soon as you talk about "on-disk" you're already limiting your
>> thinking too much, since we're not necessarily serializing to disk,
>> and the idea of having the "same representation" somewhere is somewhat
>> limiting too.  An MPI archive builds up a skeletal representation of
>> what needs to be serialized and then MPI reads it out of memory into
>> the hardware.
>>
>> For example, MPI archives support array serialization for all PODs
>> that are not pointers and do not contain pointer members.
> 
> I'm not sure I follow. How does an MPI archive serialize an array of X,
> where X is an arbitrary POD (without pointer members)?

You need to construct an MPI 'datatype', which is conceptually a record of
how the type is laid out as a sequence of (offset, nested_datatype) pairs. 
There are predefined datatypes for various types, including arithmetic
builtins.  MPI provides some functions to assemble datatypes corresponding
to structures and arrays, including strided arrays.  When the message is
sent somewhere, the receiver needs to provide a compatible datatype.  This
requires only that it has the same sequence of basic types, the offsets can
be different.  So you could, for example, build a datatype that would read
directly from a std::list, and read it back somewhere else as a vector, or
even a struct.  And vice versa.

The 'usual' way (if there is such a thing; datatypes seem to be not used
much in MPI) of constructing datatypes is using offsetof(), or just knowing
what the layout is for compiler X, and constructing the datatype by hand. 
I don't know what mechanism Dave is thinking of to construct the datatype,
it sounds unlikely that it could be done via a usual serialization function
(but maybe if you could do member pointer arithmetic to replace
offsetof() ?).  I don't understand the restriction to PODs with no pointers
though, as pointers are no problem - at least in principle - you just
recursively follow the pointer when constructing the typemap.  There is a
difference though, as a datatype for an object without nested pointers has
fixed offsets, whereas for a type containing pointers a new datatype needs
to be constructed for each object that is serialized.

Cheers,
Ian