Re: [boost] [serialization] fast array serialization (10x speedup)

From: Matthias Troyer (troyer_at_[hidden])
Date: 2005-10-19 12:12:02

On Oct 19, 2005, at 7:55 AM, Robert Ramey wrote:

> David Abrahams wrote:
>
>> "Robert Ramey" <ramey_at_[hidden]> writes:
>>
>>
>>>> Why should anyone still use the other version? To save the compile
>>>> time for 5 lines of code?
>>>>
>>>
>>> LOL - believe me, someone will want to do it differently.  I can't
>>> say how or why - but believe me it will happen.  The adaptor
>>> approach lets everyone add their own thing and lets everyone else
>>> pick and choose which combination of things they want to add.
>>>
>>
>> IIUC, you currently have some default implmentation that's 10x slower
>> than the one Matthias is proposing.  Is there any good reason that  
>> the
>> fast implementation shouldn't be the default?
>>
>
> The current implemenation is universal.  The fast ... archive will  
> only make
> a difference on those collections whose storage is contiguous.  Its  
> not even
> clear to
> to me that std::vector is storage is guarenteed to be contiguouse.

It is guaranteed ito be so by the standard.

> Its not
> clear that
> its even applicable to archives other than native binary

Robert, in one of my previous e-mails I listed FOUR very different  
types of archives that need this feature, and not only for  
performance, but also for memory reasons. The MPI archive without  
support for array serialization in addition need unacceptably large  
amounts of memory (up to 5 times the storage needed for the array  
when serializing arrays of int on 64 but machines).

> Basically Mattias enhancement is an exploitation of a special  
> cases.  That's
> actually
> a very good idea.  But that's not the same replace of the general  
> solution
> with something
> more intricate and more fragil - at no improvment in performance.

It's not a special case, since many archive types will directly  
support serialization of contiguous arrays, and some need it to be  
feasible at all.

Regarding fragility: having to implement serialization of  
multi_array, ublas arrays, MTL, Blitz ... inside a fast array archive  
is much more fragile (if it were feasible at all, see my other mail),  
since changes to small implementation details of those libraries will  
break the code. There is no such danger with the std::vector  
serialization and C-style arrays.

Matthias