Re: [boost] Re: generic polyvalent input/output ['pickle']

From: Beman Dawes (beman_at_[hidden])
Date: 2000-11-21 15:03:10

At 12:01 PM 11/21/2000 +0000, Jose Orlando Pereira wrote:

 >I'm sorry to only now join the discussion, but I was not
 >subscribed to this list before.

Glad you could join us.

 > ...
 >I'll try to paint the big picture: As I see it, at the higher level,
 >XTL and current version of persistence are very similar. For instance:
 >
 > class X {
 >   int i, j;
 >   list<int> l;
 >
 >   template<class Desc>
 >   void describe(Desc & descriptor) {
 >    descriptor & i & j & l;
 >   }
 > }
 >
 >versus:
 >
 > class X {
 >   int i, j;
 >   list<int> l;
 >
 >   template <class Stream>
 >   void composite(Stream& stream) {
 >     stream.simple(i).simple(j).simple(l);
 >   }
 > }
 >
 >However, I prefer the more verbose version in XTL. ...

Seems too dense to me, but that is a matter of personal taste.  I think 
Kevlin's suggestion is a bit better, though:

   descriptor << i << j << l;

"describe" seems a bit more to the point than "composite", but as always 
the names are placeholders until more important issues are nailed down.

I have some concerns about the use of describe/composite member function 
templates:

   * What happens when you need several different views of the same class?
   * Does a member function couple the class design too closely to the 
persistence/externalization library?

 >Annotating types is necessary as in C++ (due to its C legacy)
 >some types have more than one semantics. The usual example
 >is the pointer, which might be an array, an option item (wether
 >NULL or not), a reference, a linked strucure, etc...

Yes, good point.  Becomes part of the requirements, if I understand 
correctly (IIUC).

 >XTL syntax easily accomodates this:
 >
 > class Y {
 >   int* i;  // may be NULL
 >   int* j;  // must not be NULL
 >
 >   template <class Stream>
 >   void composite(Stream& stream) {
 >     stream.optional(i).reference(j);
 > }
 >
 >Of course, this could also be accomodated in persistence's syntax by
 >defining wrapper classes. (BTW, externalization should be targeted for
 >more than persistence: distributed programming is also important!;)

Good point.  While the presenting problem has been persistence, other kinds 
of externalization are obviously really, really, important in certain 
applications.  So if general externalization doesn't add hugely to the 
complexity...

 >In short, we mostly agree on the interface for describing
 >classes. As such we might target first an "class descritpion
 >library" and jointly propose that as a Boost library, if we
 >sort out the operator overloading issue.
 >Notice that such layer could be used for other purposes,
 >as I mention in the XTL documentation white-paper.

Interesting. For those who didn't read the paper (or like me, missed the 
point), could you give an example or two of other uses?

 >Back to externalization (pickling? marshalling? serialization?;),
 >the second issue are the layers used in the conversion. In
 >this regard I have designed XTL with separate layers for
 >format and buffer management because:
 >
 > - as Asger Nielsen explained to avoid the X*Y growth of
 >   required laryers;

In terms of Jen's persistence2 design, you would suggest something like 
Reader and Writer be parameterized by Buffer.

That brings up another question.  XTL templates use three key concepts: 
Buffer, Format, and Stream.  Is there any documentation that specifies the 
requirements of these concepts?  How different is Buffer from 
Stream?  Would it be possible to specify Buffer in a way that standard C++ 
iostreams met the Buffer requirements?  Just a thought, probably showing my 
lack of understanding.

It also seems to me that the XTL and persistence2 Format concept 
requirements are very similar.  Is that correct?

 >...
 >
 >Finally, regarding the wish to support XML and other textual formats.
 >I think that the issues is not wether we use a *implicitly typed* textual
 >format can be supported (as XTL has supported a lisp-ish format in the
 >past)
 >but wether an *explicitly typed* (regardless of textual or not) external
 >representation of data should be supported.

Have to think about that some more.
 >
 >In my opinion, suporting explictly typed external representations is
 >suficiently different to not be easily accomodated by an high level
 >interface as we are targetting (i.e. the composite/describe method).

Anyhow, thanks for joining boost!  Perhaps we can converge on a joint 
design.  We need to look at things like ease-of-use issues; persistence2 
includes some nice convenience wrappers, for example.

--Beman