Re: [boost] [serialization] fast array serialization (10x speedup)

From: Matthias Troyer (troyer_at_[hidden])
Date: 2005-11-22 12:16:00

On Nov 22, 2005, at 5:33 PM, David Abrahams wrote:

> "Robert Ramey" <ramey_at_[hidden]> writes:
>
>> I have one question about this.
>>
>> What is the ultimate purpose.  That is it just to optimize
>> serialization of certains types of collections of bit streamable
>> objects? or does have some more ambitious goal.
>
> I thought I highlighted the ultimate purpose quite clearly already:
>
>   ,----
>   | For many archive formats and common datatypes there exist APIs
>   | that can quickly read or write contiguous sequences of those types
>   | all at once (**).  Reading or writing such a sequence by
>   | separately reading or writing each element (as the serialization
>   | library currently does) can be an order of magnitude more
>   | expensive.
>   `----
>
>   We want to be able to capitalize on the existence of those APIs, and
>   to do that we need a "hook" that will be used whenever a contiguous
>   sequence is going to be (de)serialized.  No such hook exists in
>   Boost.Serialization.
>
>   (**) Note that this capability is not necessarily tied to bitwise
>        serialization or the use of a binary representation.
>
> In particular, I took special pains to clarify above (**) that this is
> *not* merely about "serialization of certains types of collections of
> bit streamable objects."

To be a bit more specific, and give one example, an MPI message  
passing library will convert the binary representation of data when a  
message is sent between machines with incompatible binary formats.  
This is then more than just streaming the bits. This is all hidden by  
the API though, and we do not worry about it.

Matthias