Re: [boost] [serialization] fast array serialization (10x speedup)

From: Peter Dimov (pdimov_at_[hidden])
Date: 2005-11-23 08:31:26

David Abrahams wrote:
> "Peter Dimov" <pdimov_at_[hidden]> writes:
>
>> Conceptual issues with (2) aside (the external format of X is
>> determined by X itself and you have no idea whether the structure of
>> X[0] also describes X[1]),
>
> I don't know what you mean.

When your mpi_archive is given an arbitrary X, there is no way to know 
whether serializing an array of X can be transformed into serializing a X 
and then building an array datatype out of the result. That's because X's 
serialize functions can do _anything_. Your archive is simply not allowed to 
alter them. (*)

That is why the optimization must be explicitly enabled by the author of X. 
My approach deals with that by allowing him to write a save_sequence 
overload for X:

void save_sequence( mpi_archive & a, X const * x, size_t n )
{
    a.save_pod_array( x, n );
}

You are trying to generalize this by allowing X's author to define a trait 
that isn't archive specific. I claim that this doesn't work, because whether 
an array of X supports optimized writes into an archive A is (in my 
experience) determined by the combination of X and A, and nothing is gained 
by defining separate traits with the hope that another archive might reuse 
them (YAGNI principle).

In short, you'll end up defining an is_mpi_pod trait, give it another, 
non-MPI name, and pretend that it's generally useful. It's not. It's MPI 
specific.

(*) Consider a X that is essentially a discriminated union as an example.

struct X
{
    int valid_field_; // 0..1
    int field1_;
    double field2_;
};

that serializes valid_field_ and then either field1_ or field2_ depending on 
valid_field_.