Re: [boost] [serialization] Enhancing Archives with Archve Adaptors

From: Robert Ramey (ramey_at_[hidden])
Date: 2005-11-27 12:57:24

Ian McCulloch wrote:

> Would it be correct to describe the adaptor's function as overriding
> serialization for a particular type (or set of types) and serializing
> it in
> a different format, using facilities already available to the
> archive(s) ?

Its a little more broad than that.  It would permit ANY code
to be attached to a specified combination of archve-datatype.

Just to speculate on an imaginary example - please don't treat
this as a serious proposal which I have to defend.  Suppose
someone comes along and looks at the xml_archive.  He says
wow - now that is way cool - But its really not done.  What I
need is to create an xml schema along with with my xml_archive
so i can use my xml wizebang tool to browse and maybe
edit my archive!.  Damn - that means I have to make my
own implementation xml_archive with makes special
versions name/value pair serialization.  Oh that's not that
bad. I can just derive from the current xml_archive and
add my overrides there.  Damn - it turns out that I have
TWO current xml_archives - one for wide characters and
wide stream i/o and one for narrow with characters.  OK
I can fix up the the base class.  Damn - that's a lot of tricky
code - I don't want to mess that.  OK I can insert a new
base class above xml_oarchive.  Damn - now I"ve changed
xml_oarchive for existing archives.  Which might be OK
or might not - in any case now someone has to maintain
a more elaborate version of xml_archive which now inherits
code for a special case - creating of xml_schema along
side of the archive.  And not everyone else wants this
extra feature so I need to add tothe documentation to show
how to turn it on and off and specify the file name which
should be applied for it and accomodate different variations
of xml_schemas.  So now every user of xml_archive has
to go review an expanded interface for features which
most likely interest him.  Now that its "part of the library"
someone has to field questions on this most likely from
people who will come to conclude that they don't need it.

Now comes along the guy with the "next" great feature.  He
want's to move the collection count into the tag rather than
as the first data member.  He really needs this.  Now the
whole cycle starts all over again.  In no time at all, the
xml_archive has all the "required" enhancements but
now they are interacting - and writing overrides for the
serialization functions is getting almost impossible.  Of
course, by this time, saving of collections also include
save_array so its all mixed in there.

Now compare this with the "archive adaptor" concept.

One archive adapter - xml_schema_adaptor which
adds code to create an xml_schema alongside of the
original xml_archive.  Of course the author of this
adaptor has to specify enhanced versions of serialization
code for certain types - but that's what he wanted to
do in the first place.  He applies his adaptor to each
of the two existing xml_archives xml_oarchive and xml_woarchive
and thus creates two new ones - xml_with_schema_oarchive
and xml_with_schema_woarchive (note the "w").  Now
he documents his two NEW archives - or perhaps his
adaptor with the new calls they have - atlease a new
constructor and probably more.  (Note these two
new archive classes may not be able to read the
archive data created by original xml_archive class.
This is to be expected as they implement new
functionality).

So everyone has what he want's.

Override functions for special
combination of archive and type are written only once
by the person who wants/needs them - so he's happy

The library user who want's to just make a minimal xml_oarchive
is no more unhappy than he is now.  Nothing has changed for
him.

Users who want the new functionality get it and have a
document which describes the functionality as an extension
to the original xml_archive.  Much easier to digest as
well as write.

Users who have made there own extensions to the xml_oarchive
can now "mix-in" the new schema generation into their own
archive with just a little bit of boiler plate code.

The library maintain has no more work to do.  The author
of the adaptor is happy to maintain his code which is small
and depends only upon the quite "narrow" interface of the
core serialization library.  Furthermore, the author of the
extension might be pleased to find that more people
are using his extension as it can be applied to archives
of which he had no knowledge.

> As far as I can tell, what distinguishes the 'adaptor' case from the
> array extensions that have been discussed at length, is
>
> 1. In the default case the save_array() function (**) reproduces the
> existing behaviour.

> That is, an array_adaptor<Base> would produce a
> bit-for-bit identical archive to using the Base archive type itself,
> for all Base archives that currently exist in the serialization library.

Hmm - I'm not sure what you mean by this but I can say the following:

I believe that new the archive produced by the application 
bitwise_archive_adaptor
to binary_oarchive would produce a bit for bit identical archive as
the current binary_oarchive does.  But in the face of varying compilers
configuratons and lots of small things I really couldn't say for sure
without undergoing a tremendously tedieous examination at a very low level.

Of course, if the bitwise_archive_adaptor where applied to something
like a text_oarchive it would result in something quite different.
const float x[2] = {1.0, 2.5}
ar << x;

Now looks like 1.0 2.5 in a text archvive.  The archive class which
results from application of the bitwise_archive_adaptor would
use save_binary to produce an equivalent (for this platorm)
string of characters in base64 code.  So it would be odd to
me that someone might want to do this.

> 2. In the non-trivial case where save_array() does something
> different to
> the default, it needs to invoke functionality that does _not_ already
> exist in the serialization library, and is _archive_ _specific_.

correct.

> Both of these points strongly suggest, to me at least, that the
> save_array() extension is not properly an adaptor.  Do you have a
> different interpretation?

I've attempted to illustrate that it can be implemented as an adaptor
and doing so will have certain benefits over alternatives.  In the
particular case we've been discussiing all he high_performance
computing archives will be derived from a common base class
so my adaptor idea isn't required.

But then, someone is going to insist that the old binary_oarchive
really needs this enhancement.  Since save_array would now
be part of the high_performance computing archives base class
it wouldn't be available to others.  However, my adaptor
would be available to mix in for anyone who feels they
have to have it.  Of course I only know about binary_oarchive.
For all I know someone out there has made his own
derivation or variation of binary_oarchive.  But I don't have
to know - my adaptor can be use by them if they want.

Here is the key point.  I'm not really concerned about specifically
about save_array.

The save_array optimization is one example of any number
of enhancements and/or extentions that people might want to
make.  But it is not the only example.  We can't go mixing
every great idea into the core library without running into
an intractible scalabilty problem.

Robert Ramey