Re: [boost] generic polyvalent input/output ['pickle']

From: Jens Maurer (Jens.Maurer_at_[hidden])
Date: 2000-11-22 18:08:11

I'll try to address a few concerns in this message, all in one
instead of replying individually.

 - versioning (Asger Alstrup): The example should work 1:1
with the persistence library.
 - Buffer template vs. iostream: I need to code a fixed_size_streambuf
and have some speed comparisons. (TODO)
 - XDR: I've now got a quick hack implementation of the XDR features
required by my test program.  See discussion below.  Btw, XTL's
xdr.h seems to fail silently for 64bit long.
 - I'm happy with overloading operator<< .  Note that this will
also be used for reading, i.e. it will (sometimes) change the variables
i and j in this example:
  descriptor << i << j;
so it looked a bit counter-intuitive to me on first sight.
 - Yes, the infrastructure can also be used for network
communications etc.
 - Kevlin Henney's wrapper idea for reference() etc. look nice.
 - Just to make sure: The whole thing is implicitly typed in
all cases, i.e. the program must know the types before it can
start reading.  This is why we have templates all over.
 - class member functions are too restrictive (Beman): The
persistence library uses this call chain in all cases:  save
(or load) -> describe (global function) -> describe (member
function)
You can overload at any stage.  So anyone can use a non-intrusive
global function instead of a member function.
 - Several views on the same data structure (Beman):  This
requires a new type so that overloading selects the correct
load/save function:
  class verbose_vector {
    verbose_vector(const std::vector<int> &v) : ref(v) { }
    const std::vector<int> & ref;
  };
  // add save/load functions to taste

Alternatively, you could program another Reader/Writer, depending
whether your "different view" is structural or presentational.
  

The discussion so far has focused very much on the describe()
approach.  (I've shamelessly stolen the implementation idea
from XTL, but I think it's the correct way of doing it, except
for a few syntax issues.)

The describe() approach basically says that an object is
exactly the sum of its parts.  This is mostly adequate for
C-style struct's.  However, I would expect that the externally
visible state of a full-blown C++ object is considerably
different from its internal representation. For example, having
internal pointers into some internal buffer usually means writing
and reading offsets (see the example in the XTL documentation).
For a real-world example, imagine saving and loading a
boost::reg_expression (see the boost regex library.)  
In this case, the describe() approach is inadequate.

In short, we have:
 - describe() for "easy" C-style struct's and similar
 - Separate save()/load() for complex C++ objects (where load()
is actually similar to a constructor).   This case should not
be ignored.

Writing separate save()/load() functions is harder and/or
more redundant than writing a single describe() function.

I understand that XTL has a few features so that more stuff
can be handled using the describe() way.

Completely disparate from this discussion is the Buffer/iostreams
discussion.

All in all, this probably means splitting off more parts from the
persistence library, with clearly specified interfaces:

 - Formatting of elementary items (my Reader/Writer, XTL's Format).
XDR and GIOP libraries on its own would come in handy here (XTL's
xdr.h looks slightly messy and somewhat non-portable to me, though).
The interface is: reading and writing int's, double's, strings,
structure and sequence boundaries etc.
 - Buffer/iostream: Needs more investigation.

Jens Maurer