Re: [boost] [serialization] Enhancing Archives with Archve Adaptors

From: David Abrahams (dave_at_[hidden])
Date: 2005-11-27 09:49:59

"Peter Dimov" <pdimov_at_[hidden]> writes:

> Robert Ramey wrote:
>
>>>  I actually don't want to get into a discussion of which
>>>  non-intrusive design is best.  The social and code interoperability
>>>  dynamics of any non-intrusive design are the same, and that's
>>>  really what I want to discuss.  Please let me know when you're
>>> ready to talk about that.
>>
>> Sorry,  I don't even know what that means.
>
> Social dynamics aside, one point is that any intrusive design, no
> matter how clever, will always be more intricate and complex than it
> needs to be if the library doesn't provide a bit of help.

Peter, 

Robert has many times expressed unwillingness to consider an intrusive
design.  I understand that you're trying to help, but intrusive
designs have been proposed many times in the past (starting at least 3
years ago!), and every time, Robert has dug in his heels further
because he believes it's the first step down the road toward an
unmaintainable library.  I think it's high time we respect his
concerns and find a way to get fast array serialization that doesn't
violate his non-intrusiveness constraint.

,----
| That's why our current design, described in
| http://lists.boost.org/Archives/boost/2005/11/97002.php, makes no
| changes to the Serialization library.
`----

> But let's put complexity aside for a moment and just think about the
> problem from the perspective of a programmer that needs to provide
> serialization support for his class X.
>
> It so happens that X holds a contiguous array of char. The
> programmer knows that for some archives it can be a major
> performance gain to invoke their char[]-writing method, but he
> doesn't know which these archives are. Some of them may not have
> been written yet.
>
> So we need to give these programmers a _documented_ function that
> they can call when they serialize arrays. And it is best for this
> function to be described in the documentation of the serialization
> library, because that's what these programmers are using. They
> aren't writing archives or archive adaptors, they may not have heard
> of Dave's enhancements; they just serialize their own type.

Whether that function is going to be documented in the Serialization
library or in some other library is 100% up to Robert.

> Whether this function should be named save_array and only accept
> (pointer, size) arrays, or save_sequence and accept arbitrary
> (iterator, size) sequences (as I proposed), is a matter of debate

Maybe not a matter of debate.  I have no objection to a proposal that
uses an (iterator,size) interface as long as it handles std::vector
and any other cases not covered by the code shown in your earlier
posting.

> , but it doesn't change the fundamental point.
>
>     The serialization library should provide a function that class
>     authors can call when they serialize arrays...
>
> ... because no one else can.

I hate to contradict you because I agree with the spirit of your
argument, but of course someone else _can_ provide the function, and
it's important that we say so, or Robert will quite reasonably feel
we're forcing the conclusion down his throat.  As proof, our design
described in http://lists.boost.org/Archives/boost/2005/11/97002.php
makes no changes to the library and could be packaged as a separate
add-on library built upon Boost.Serialization.

If Robert insists that the function be provided separately, he is also
buying into a situation where this function in the add-on library has
to be used by every serialization function that _might_ be used in a
performance-critical context, and every archive choice made in what
_might_ be a performance-critical context must come from the add-on
library, if an appropriate archive exists there (I am thinking e.g.,
of binary archives that would be present in the add-on library while
text-based archives probably would not).  That's what I want him to
think about.  

If he understands what that means and prefers to avoid intrusion on
the library design anyway, Matthias and I are willing to accept that
and never bring it up again.  After three years of hammering on this
one point I can't blame Robert for being tired, and I have no reason
to believe new arguments are likely to change his mind about it.

> Now, if we add this function to the library, the next logical step
> is for the library to use its own function when serializing arrays,
> std::vector and std::valarray. It would be pretty odd not to do
> so. :-)
>
> We (*) can propose a complete list of changes to the library if you are 
> interested, so that you can evaluate its impact. The support for objects 
> without a default constructor does complicate matters a bit in the 
> std::vector case, but it can be done. Existing binary archives should 
> receive a significant speedup in the char[] case without any changes to the 
> external archive format.
>
> (*) Hopefully the rest of the "we" agrees. 

In principle, yes, but in practice, no.  Since Robert has made it very
clear that he doesn't want to consider any intrusion on the library
design, I can't join in such a proposal.  The best I can hope for is
that he understands the consequences of his choice and feels
comfortable with what other people are planning to do with his
library.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com