Re: [boost] [serialization] fast array serialization (10x speedup)

From: Ian McCulloch (ianmcc_at_[hidden])
Date: 2005-11-22 15:54:43

Peter Dimov wrote:

> Ian McCulloch wrote:
>> Peter Dimov wrote:

[snip]

>>> So what's all the fuss about?
>>
>> That isn't quite all that needs to be done.
> 
> You are right, a std::vector needs to be special-cased to use a pointer.
> 
>> (1) minor nit: an interface that uses (iterator, size) would be
>> better than
>> a container-based algorithm because that would make it easier to do
>> optimizations based on the iterator type (eg, memcpy, or MPI
>> operations in the case of a pointer, or maybe some kind of
>> distributed iterator in combination with a parallel IO library?).
> 
> I don't understand.

If you are going to overload something, it is easier to overload on a type
that actually appears in the function signature rather than doing a
dispatch based on a nested ::iterator type.  Anyway this is moot because I
misunderstood you to mean that save_collection would be the customization
point, but you apparantly intended save_array instead.  sorry.

> 
>>> template<class Archive, class It>
>>> inline void save_sequence(Archive & ar, It it, unsigned count)
> 
> looks decidedly (iterator, size) based to me.
> 
>> Also, the collection isn't necessarily in the form of a container
>> (although a proxy container would probably suffice for that case, and
>> come to think of it, to handle resizing the container on load it
>> might actually be preferable).
> 
> I don't understand this either.

For example, if you had a bare array that you managed yourself with array
new/delete, you would need to make some kind of proxy container to pass it
to save_collection.  But on loading the array you need to do a resize
somewhere, and a proxy container would be one way to handle that (the
resize() member of the proxy would handle the memory reallocation).

> 
>> (2) another minor nit: it is probably more convenient to handle the
>> details of save_sequence() inside the archive (similarly to other
>> primitive types), rather than as a free function.
> 
> The point is that you can overload the free function inside your archive's
> namespace.

Ok, but as Dave pointed out the lookup rules complicate this approach.

> 
>> (3) : save_collection() [or some functional equivalent] isn't part of
>> the public interface of the serialization library.  For whatever
>> reason this seems to be the sticking point.
> 
> save_collection isn't - and probably shouldn't - but save_sequence would
> be.

Fine.  Anything that allows customization of arrays/sequences.  The actual
interface is a detail at this point.

Cheers,
Ian