Re: [boost] [serialization] fast array serialization (10x speedup)

From: David Abrahams (dave_at_[hidden])
Date: 2005-11-24 12:31:04

"Peter Dimov" <pdimov_at_[hidden]> writes:

>> What if the number n is infinite (e.g. all possible structs
>> consisting only of fundamental types), which is what Robert calls
>> "bitwise serializable"?
>
> Structs aren't bitwise serializable in general because of
> padding/packing/alignment. Archives that do not have a documented
> external format and just fwrite whatever happens to be in memory at
> the time aren't really archives, they are a very specific subset
> with limited uses (interprocess communication on the same machine,
> the same compiler and the same version) that should not shape the
> design. 

If Stepanov had used that philosophy we wouldn't have algorithms
specialized for random access iterators.  Containers with random
access are "a very specific subset" with, arguably, "limited uses"
(don't forget that the original Lisp guys thought it would be better
if everything were made up of cons cells).  The ability to specialize
generic algorithms to take advantage of special properties of
"specific" datatypes is fundamental to Generic Programming.

In fact, every example we can think of so far where optimized array
serialization is useful is just such a "specific" archive.

> ("Archive" implies persistency, and relying on a specific
> memory layout is not a way to achieve it.)

I think Robert's statement

  "Here, we use the term "serialization" to mean the reversible
  deconstruction of an arbitrary set of C++ data structures to a
  sequence of bytes. Such a system can be used to reconstitute an
  equivalent structure in another program context. Depending on the
  context, this might used implement object persistence, remote
  parameter passing or other facility. In this system we use the term
  "archive" to refer to a specific rendering of this stream of
  bytes. This could be a file of binary data, text data, XML, or some
  other created by the user of this library. "

defines the intention of the library and if "archive" implies
persistency to you it was perhaps an unfortunate naming choice, but I
don't think that should be used to make arguments about the design
that contradict his intention.

> If you have such an archive, you can add an overload SFINAE'd on 
> is_bitwise_serializable 

Robert wants portability to vc6, which doesn't support SFINAE.  I
doubt he'd want to accept a change that, to be practically taken
advantage of, would require users to apply SFINAE.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com