Re: [boost] [serialization] fast array serialization (10x speedup)

From: Martin Slater (mslater_at_[hidden])
Date: 2005-10-10 07:09:02

> 
> ---- SUMMARY ---------
> 
> If I may summarize this solution as follows:
> 
> template<class Base>
> class fast_oarchive_impl :
>      public Base
> {
> public:
> ...
>      // custom specializations
>      void save_override(const std::vector<int> & t, int){
>          save_binary(t, sizeof(int) * t.size());
>      }
> 
>      // here's a way to do it for all vectors in one shot
>      template<class T>
>      void save_override(const std::vector<T> & t, int){
>          save_binary(t, sizeof(T) * t.size());
>          // this version not certified for more complex types !!!
>          BOOST_STATIC_ASSERT(boost::is_primitive<T>::value);
>          // or pointers either !!!
>          BOOST_STATIC_ASSERT(boost::is_pointer<T>::value);
>      }
> 
> ...
> };
> 
> then I see several major disadvantages of this approach:
> 
> 1.) it fixes the value types for which fast array serialization can  
> be done

I worked around this by introducing an intermediary type 
ArchiveByteArray as follows

     struct BinaryArchiveByteArrayI
     {
         int count;
         void *ptr;
     };

     template<class T>
     BinaryArchiveByteArrayI MakeArchiveInputByteArray(int count, T *t)
     {
         BinaryArchiveByteArrayI res;

         res.count = count * sizeof(T);
         res.ptr   = t;

         return res;
     }

     struct BinaryArchiveByteArrayO
     {
         int count;
         const void *ptr;
     };

     template<class T>
     BinaryArchiveByteArrayO MakeArchiveOutputByteArray(int count, const 
T *t)
     {
         BinaryArchiveByteArrayO res;

         res.count = count * sizeof(T);
         res.ptr   = t;

         return res;
     }

BOOST_CLASS_IMPLEMENTATION(BinaryArchiveByteArrayI, primitive_type);
BOOST_CLASS_IMPLEMENTATION(BinaryArchiveByteArrayO, primitive_type);

Then adding in the input / output archive

         void load(BinaryArchiveByteArrayI &ba);
         void save(const BinaryArchiveByteArrayO &ba);

This should be able to cope with array types and anything else that 
takes a contiguous block of memory. Then there is one set of overloads 
per type , ie for std::vector

     template<class U, class Allocator>
     inline void load(
         BinaryInputArchive & ar,
         std::vector<U, Allocator> &t,
         const unsigned int /* file_version */
     ){
         boost::mpl::if_<boost::is_pod<U>, Detail::VectorLoadPodImp, 
Detail::VectorLoadImp>::type(ar, t);
     }

     template<class U, class Allocator>
     inline void save(
         BinaryOutputArchive& ar,
         const std::vector<U, Allocator> &t,
         const unsigned int /* file_version */
     ){
         boost::mpl::if_<boost::is_pod<U>, Detail::VectorSavePodImp, 
Detail::VectorSaveImp>::type(ar, t);
     }

This could probably have a dispatch mechanism above it checking for an 
archive trait to dispatch to either the fast or default implemention.

Martin

-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.344 / Virus Database: 267.11.14/127 - Release Date: 10/10/2005