Re: [boost] [serialization] fast array serialization (10x speedup)

From: Peter Dimov (pdimov_at_[hidden])
Date: 2005-11-22 13:40:35

David Abrahams wrote:

>  We want to be able to capitalize on the existence of those APIs, and
>  to do that we need a "hook" that will be used whenever a contiguous
>  sequence is going to be (de)serialized.  No such hook exists in
>  Boost.Serialization.

FWIW, this is what I do in my library:

template<class W, class T, class A> void write(W & w, std::vector<T, A> 
const & v)
{
    int m = v.size();

    begin_sequence(w, io::type_of(v), io::type<T>(), m);

    if(m > 0) write_sequence(w, &v[0], m);

    end_sequence(w, io::type_of(v), io::type<T>(), m);
}

The default implementation of write_sequence just does:

template<class W, class It> inline void write_sequence(W & w, It first, 
std::size_t m)
{
    for(; m > 0; ++first, --m) write(w, *first);
}

but writers that support contiguous fast writes overload write_sequence( 
my_writer&, T*, size_t ).

Looking at collections_save_imp.hpp:

template<class Archive, class Container>
inline void save_collection(Archive & ar, const Container &s)
{
    // record number of elements
    unsigned int count = s.size();
    ar << make_nvp("count", const_cast<const unsigned int &>(count));
    BOOST_DEDUCED_TYPENAME Container::const_iterator it = s.begin();
    while(count-- > 0){
        //if(0 == (ar.get_flags() & boost::archive::no_object_creation))
                // note borland emits a no-op without the explicit namespace
                boost::serialization::save_construct_data_adl(ar, &(*it), 
0U);
        ar << boost::serialization::make_nvp("item", *it++);
    }
}

all that needs to be done is:

template<class Archive, class It>
inline void save_sequence(Archive & ar, It it, unsigned count)
{
    while(count-- > 0){
        //if(0 == (ar.get_flags() & boost::archive::no_object_creation))
                // note borland emits a no-op without the explicit namespace
                boost::serialization::save_construct_data_adl(ar, &(*it), 
0U);
        ar << boost::serialization::make_nvp("item", *it++);
    }
}

template<class Archive, class Container>
inline void save_collection(Archive & ar, const Container &s)
{
    // record number of elements
    unsigned int count = s.size();
    ar << make_nvp("count", const_cast<const unsigned int &>(count));
    save_sequence( ar, s.begin(), count );
}

unless I'm missing something fundamental.

So what's all the fuss about?