[boost] RE: Serialization questions

From: Robert Ramey (ramey_at_[hidden])
Date: 2004-08-31 17:16:23

> Hello again,
> 
> I have started using your serialization library.  It is working pretty
> well, but I have run into a few small problems:
> 
> 1) No serialization for bitset: No big deal, because I can just
>    convert to ulong, but I was surprised it wasn't there.

Feel free to add it in along with a test.  I would like to see people adding
serialization bit by bit to the existing data structures as the need becomes
apparent.  We're coming to the view that these things should be added to the
data structures themselves rather than the serialization library.  I'm
trying to get myself out of the business of serializing things. 

> 
> 2) I have a text archive with a bunch of pairs of strings.  I want to
>    append another pair.  There doesn't seem to a way to append items
>    to an archive.

LOL - an interesting question that only recently - (in the last couple of
months) occurred to me.

There is no reason that you can't create an archive and just append to it as
many times as you see fit.
        Oa << t;
        ...
        Oa << t;

EXCEPT that the t (and its constituents) should have its serialization trait
set so that memory tracking is turned off.  Otherwise, copies subsequent to
the first one won't copy any data - just a pointer to the original.  

I have considered making and testing an example of such a thing, but I've
been busy.  To support this and make a good example what is really needed
is:

a) another archve construction flag - "no_tracking" to suppress memory
tracking for the archive regardless of the setting for each individual type.
b) serialization added to boost::any and/or boost::variant. Probably doable
but maybe a little tricky.

In this way one could just save willy-nilly anything to the archive and load
later.  This would be very interesting for things like database transaction
logs which want to implement rollback/rollforward and things like that.  It
would also make a dynamite log for debugging.

Again, feel free to explore this at your leisure.

> 3) similarly, if I'm reading items from an archive, there is no simple
>    way to keep reading until the end of the file.  I have to catch the
>    exception, which seems wrong for such an unexceptional
>    circumstance.  Setting a fail bit as with streams would be nicer.

Checking the fail bit on the stream before reading the next archived item
should work. It's intended to anyway.  If it doesn't work it's a bug. There
might be a little tweaking required as I didn't test for this specifically.
If you try this and it doesn't work and you find the oversight let me know
and I'll add the fix.  Better yet, make and test a small test and submit it
to me and I'll add it into the test suite.

>    However, you could take the view that you should always know what
>    is in the archive, and to have a method of failure that does not
>    throw an exception would invite errors.  That suggests that a
>    constructor flag might work (e.g. throw_on_eof default to true).

I recently discovered that one can set a flag on an input stream so that an
attempt to read past eof will throw an exception. This would leave all
options open. After learning ths I'm considering removing the code that
checks the fail bit as I would like to leave the flexibility for the user
and make the serialization library smaller. BTW the fail bit means - "the
next operation is expected to fail", NOT "the last operation failed".  This
confused me for quite a while until I actually carefully read the
documentation for stream rather than presuming what I thought was obvious.

> 4) A minor point: It would be nice to be able to construct an archive
>    with a boost::filesystem::path, instead of creating an fstream and
>    then using that to create the archive.  It is certainly nice that I
>    can attach an existing stream to an archive, but directly using a
>    path would reduce some clutter.  It would also make it a little
>    safer, because I couldn't accidently insert something into the
>    stream that was not serialized.

Feel free to make your own - it should be trivial exercise.  It should be
easy to do deriving from any of the current archives.  Polymorhphic archive
works this way.  Its one template that "adapts" any archive it's applied to.

Also the current setup is specifically designed to permit serialized data to
be embedded within some other data stream if the user requires it. I haven't
specifically tested this either.

> I have read large parts of the discussions when you were submitting
> your library, but not all.  If I have missed something, you can just
> point me to it.

Seems like you've pretty much got it.  The thing that takes special care is
the serialization of pointers through abstract base pointers.  This works
fine but there are a couple of non-intuitive aspects that can trip you up.
The manual is pretty complete.  This is mostly because I kept adding to it
as questions kept coming up on the list.  So I would hope that one can
understand and use the package without going back and trolling through the
list.

If you don't mind, I'm posting this to the list.  Your questions are
interesting, good grist for the list, and help diminish the frequency of the
same questions getting asked multiple times.

Robert Ramey