[boost] Re: Serialization questions

From: Walter Landry (wlandry_at_[hidden])
Date: 2004-09-01 20:31:08

"Robert Ramey" <ramey_at_[hidden]> wrote:
> > Hello again,
> > 
> > I have started using your serialization library.  It is working pretty
> > well, but I have run into a few small problems:
> > 
> > 1) No serialization for bitset: No big deal, because I can just
> >    convert to ulong, but I was surprised it wasn't there.
> 
> Feel free to add it in along with a test.  I would like to see people adding
> serialization bit by bit to the existing data structures as the need becomes
> apparent.  We're coming to the view that these things should be added to the
> data structures themselves rather than the serialization library.  I'm
> trying to get myself out of the business of serializing things. 

Ok.  I just wanted to make sure that you hadn't missed it by accident.
Since I've got my stuff working, I probably won't get around to
implementing bitset, though.

> > 
> > 2) I have a text archive with a bunch of pairs of strings.  I want to
> >    append another pair.  There doesn't seem to a way to append items
> >    to an archive.
> 
> LOL - an interesting question that only recently - (in the last couple of
> months) occurred to me.

<snip>

> Again, feel free to explore this at your leisure.

Hmm.  It looks like more work than I really want to put into this
project.  I think I will content myself with the simple hack I have
been using (open the file in append mode, write a space, then open the
archive with no header).

> > 3) similarly, if I'm reading items from an archive, there is no simple
> >    way to keep reading until the end of the file.  I have to catch the
> >    exception, which seems wrong for such an unexceptional
> >    circumstance.  Setting a fail bit as with streams would be nicer.
> 
> Checking the fail bit on the stream before reading the next archived item
> should work. It's intended to anyway.  If it doesn't work it's a bug. There
> might be a little tweaking required as I didn't test for this specifically.
> If you try this and it doesn't work and you find the oversight let me know
> and I'll add the fix.  Better yet, make and test a small test and submit it
> to me and I'll add it into the test suite.

The text deserialization for string doesn't check for whether the
deserialization of the size of the string succeeded.  Since the size
is undefined, that causes read for the string itself to fail.  I have
attached a patch which seems to fix that for strings only, but it
needs to be replicated for all of the other items that have a length
argument as well.

> >    However, you could take the view that you should always know what
> >    is in the archive, and to have a method of failure that does not
> >    throw an exception would invite errors.  That suggests that a
> >    constructor flag might work (e.g. throw_on_eof default to true).
> 
> I recently discovered that one can set a flag on an input stream so that an
> attempt to read past eof will throw an exception. This would leave all
> options open. After learning ths I'm considering removing the code that
> checks the fail bit as I would like to leave the flexibility for the user
> and make the serialization library smaller. BTW the fail bit means - "the
> next operation is expected to fail", NOT "the last operation failed".  This
> confused me for quite a while until I actually carefully read the
> documentation for stream rather than presuming what I thought was obvious.

>From near the top of page 598 of "The C++ Standard Library" by Nicolai
Josuttis

  After reading the last character, the flag eofbit is not yet set.
  The next attempt to read a character sets eofbit and failbit,
  because the read fails.

>From the C++ standard, I see that when you're doing something like
reading formatted input, it might set the eofbit without setting the
failbit.  But I can't see anything that says that the failbit or
eofbit must be set preemptively.  I don't see how it could, because it
has to read all of the whitespace before figuring out whether the next
read will fail.

> > 4) A minor point: It would be nice to be able to construct an archive
> >    with a boost::filesystem::path, instead of creating an fstream and
> >    then using that to create the archive.  It is certainly nice that I
> >    can attach an existing stream to an archive, but directly using a
> >    path would reduce some clutter.  It would also make it a little
> >    safer, because I couldn't accidently insert something into the
> >    stream that was not serialized.
> 
> Feel free to make your own - it should be trivial exercise.  It should be
> easy to do deriving from any of the current archives.  Polymorhphic archive
> works this way.  Its one template that "adapts" any archive it's applied to.

Ok.  This I probably do have time to do.

> Also the current setup is specifically designed to permit serialized data to
> be embedded within some other data stream if the user requires it. I haven't
> specifically tested this either.

I think I might have cause to do that.  If so, I'll holler if I find
any bugs.

> > I have read large parts of the discussions when you were submitting
> > your library, but not all.  If I have missed something, you can just
> > point me to it.
> 
> Seems like you've pretty much got it.  The thing that takes special care is
> the serialization of pointers through abstract base pointers.  This works
> fine but there are a couple of non-intuitive aspects that can trip you up.
> The manual is pretty complete.  This is mostly because I kept adding to it
> as questions kept coming up on the list.  So I would hope that one can
> understand and use the package without going back and trolling through the
> list.
> 
> If you don't mind, I'm posting this to the list.  Your questions are
> interesting, good grist for the list, and help diminish the frequency of the
> same questions getting asked multiple times.

No problem.

Walter Landry
wlandry_at_[hidden]

--- /home/boo/arx/arx/src/boost/{arch}/++pristine-trees/unlocked/boost/boost--arx/boost--arx--1.31/wlandry_at_ucsd.edu--arx/boost--arx--1.31--patch-2/boost/archive/impl/text_iarchive_impl.ipp	2004-09-01 13:06:33.000000000 -0400
+++ /home/boo/arx/arx/src/boost/boost/archive/impl/text_iarchive_impl.ipp	2004-09-01 13:12:52.000000000 -0400
@@ -39,13 +39,17 @@
     size_t size;
     * this->This() >> size;
     // skip separating space
-    is.get();
-    // borland de-allocator fixup
-    #if BOOST_WORKAROUND(_RWSTD_VER, BOOST_TESTED_AT(20101))
-    if(NULL != s.data())
-    #endif
+    if(is)
+      {
+        is.get();
+        // borland de-allocator fixup
+        #if BOOST_WORKAROUND(_RWSTD_VER, BOOST_TESTED_AT(20101))
+        if(NULL != s.data())
+        #endif
         s.resize(size);
-    is.read(const_cast<char *>(s.data()), size);
+        if(is)
+          is.read(const_cast<char *>(s.data()), size);
+      }
 }
 
 #ifndef BOOST_NO_INTRINSIC_WCHAR_T