Re: [boost] [serialization] fast array serialization (10x speedup)

From: David Abrahams (dave_at_[hidden])
Date: 2005-11-22 15:04:08

"Peter Dimov" <pdimov_at_[hidden]> writes:

> all that needs to be done is:
>
> template<class Archive, class It>
> inline void save_sequence(Archive & ar, It it, unsigned count)
> {
>     while(count-- > 0){
>         //if(0 == (ar.get_flags() & boost::archive::no_object_creation))
>                 // note borland emits a no-op without the explicit namespace
>                 boost::serialization::save_construct_data_adl(ar, &(*it), 
> 0U);
>         ar << boost::serialization::make_nvp("item", *it++);
>     }
> }
>
> template<class Archive, class Container>
> inline void save_collection(Archive & ar, const Container &s)
> {
>     // record number of elements
>     unsigned int count = s.size();
>     ar << make_nvp("count", const_cast<const unsigned int &>(count));
>     save_sequence( ar, s.begin(), count );
> }
>
> unless I'm missing something fundamental.
>
> So what's all the fuss about?

1. Robert has expressed deep reluctance to change any part of the
   existing library, which is why we're now presenting a design that
   avoids touching it.

2. This wouldn't work well for std::vector, since we know its elements
   are contiguous but we don't know that its iterators are pointers.
   Yes, I know there are some nasty hacks that will usually work for
   getting back to pointers, but they're nasty and don't always work.

3. An archive can normally only apply an array optimization to a
   particular subset of types.  This subset varies by archive type and
   can usually be captured by a type trait such as is_POD or
   is_fundamental.  We'd like to encapsulate that choice in a base
   class template that allows us to avoid writing complex dispatching
   logic in each array-optimized archive.  Partial ordering rules make
   that impossible with the above design, because the save_sequence
   above will be a better match than one that operates on
   some_base<Archive>.

4. The fuss is really about what happens when you have a design that
   doesn't insert the equivalent of a save_sequence hook in the
   serialization library.  It's a social/library-interoperability
   phenomenon that I haven't even had a chance to discuss yet -- and I
   really don't want to until Robert has had a chance to digest our
   design and understand where the speedups can come from.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com