Re: [boost] [serialization] fast array serialization (10x speedup)

From: David Abrahams (dave_at_[hidden])
Date: 2005-11-25 13:44:53

"Robert Ramey" <ramey_at_[hidden]> writes:

> David Abrahams wrote:
>> "Robert Ramey" <ramey_at_[hidden]> writes:
>>
>>> To summarize how we arrived here.
>>> =================================
>>
>>  <snip>
>>
>>> e) So it has been proposed binary_iarchive be re-implemented
>>> in the following way
>>>
>>> iarchive - containg default implementation of load_array
>>>   binary_iarchive - ? presumablu contains implementaion of load_array
>>>     in terms of currently defined load_binary
>>>
>>> Its not clear whether all archives would be modified in
>>> this way or just binary_iarchive.
>
>> This is extremely discouraging.  After I stated many times that our
>> design had been changed so as NOT to modify any code in the
>> serialization library, after we put the array-optimized archives in a
>> separate sub-namespace so that they could live alongside the existing
>> ones in the library, after I offered to put all of the code in some
>> remote part of Boost not associated with the serialization library,
>> you state that we are proposing to change the serialization library
>> code.
>>
>
> I was referring to :
>
>> archive/array/binary_iarchive.hpp
           ^^^^^

Note the directory element.  This is a completely different file from
any in the serialization library.  Also note that I said up front that
I was leaving out namespaces in my synopsis.  Can I infer from this
that you really had no idea that the code in this file was to go in
namespace boost::archive::array?

If it really looked to you as though we were proposing to change the
library, in the face of all our statements to the contrary shouldn't
you have at least asked us for an explanation?

>> ..................................
>>
>>  class binary_iarchive
>>    : public array::iarchive<
>>          array::binary_iarchive
>>        , archive::binary_iarchive_impl<binary_iarchive>
>>      >
>>  {
>>      template <class S>
>>      binary_iarchive(S& s, unsigned int flags)
>>        : binary_iarchive::iarchive_base(s,flags)
>>      {}
>>
>>      // use the optimized load procedure for all fundamental types.
>>      typedef boost::is_fundamental<mpl::_>
>>        use_array_optimization;
>>
>>      // This is how we load an array when optimization is appropriate.
>>      template <class ValueType>
>>      void load_array(ValueType * p, std::size_t n, unsigned int
>>      version) {
>>          this->load_binary(p, n * sizeof(ValueType));
>>      }
>>  };
>>
>
> I'm also presuming - maybe incorrectly -  that the serialization for
> something like a C++ array would contain code to invoke
> load_array.

That code is in array::iarchive.  That should have been clear from the
comment on its load_override member function.

        // Load T[N] using load_array
        template<class T, std::size_t N>
        void load_override(T(&x)[N], unsigned int version);

> So if I re-compile an existing application and use it to load
> a binary_archive created under the previous version of the library
> the function load_array will be invoked.  The same data
> will have been previously serialized with a loop of serializing
> each data member.  Without knowing when load_array is
> invoked, one can't know for sure with the old archives will
> in fact be readable.   Verifying that previously existing archives
> of type binary-?archive will be readable will be a non-trivial task.

This can't really be a serious objection, can it?  A program would have
to be changed to #include different files and use different namespaces
in order for any difference to be observed.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com